Jul 27 23:34:56.810044 ip-10-0-128-69 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 27 23:34:56.810058 ip-10-0-128-69 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 27 23:34:56.810068 ip-10-0-128-69 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 27 23:34:56.810389 ip-10-0-128-69 systemd[1]: Failed to start Kubernetes Kubelet. Jul 27 23:35:06.968277 ip-10-0-128-69 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 27 23:35:06.968294 ip-10-0-128-69 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 89e262411e1a49a19d30a7c77363fad7 -- Jul 27 23:37:04.532820 ip-10-0-128-69 systemd[1]: Starting Kubernetes Kubelet... Jul 27 23:37:04.892929 ip-10-0-128-69 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:37:04.892929 ip-10-0-128-69 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 23:37:04.892929 ip-10-0-128-69 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:37:04.892929 ip-10-0-128-69 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 23:37:04.892929 ip-10-0-128-69 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:37:04.893862 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.893743 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 23:37:04.899421 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899404 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.899421 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899421 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899426 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899430 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899433 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899436 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899440 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899442 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899451 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899454 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899456 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899459 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899462 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899464 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899468 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899470 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899473 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899476 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899478 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899481 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899483 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.899483 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899486 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899490 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899492 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899495 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899498 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899501 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899503 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899506 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899509 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899523 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899526 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899529 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899532 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899534 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899537 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899539 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899542 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899544 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899547 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.900074 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899549 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899552 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899555 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899557 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899560 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899562 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899565 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899567 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899570 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899572 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899575 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899577 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899579 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899582 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899585 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899588 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899591 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899593 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899597 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899600 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.900541 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899603 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899605 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899608 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899610 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899613 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899616 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899618 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899621 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899623 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899626 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899629 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899631 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899634 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899636 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899639 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899641 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899644 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899646 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899649 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899651 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.901210 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899653 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899656 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899658 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899661 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899663 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899666 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899669 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899674 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899679 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899684 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899688 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899691 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899694 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899697 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899700 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899703 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899705 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899708 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.901920 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.899710 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900405 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900411 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900414 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900416 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900419 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900422 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900425 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900427 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900430 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900433 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900435 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900438 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900440 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900443 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900445 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900448 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900450 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900453 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900455 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.902397 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900458 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900461 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900464 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900467 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900469 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900472 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900475 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900477 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900480 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900483 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900485 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900488 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900490 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900493 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900495 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900498 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900500 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900502 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900505 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900507 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.902910 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900510 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900527 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900530 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900533 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900536 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900538 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900542 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900546 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900550 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900553 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900557 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900560 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900563 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900566 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900569 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900572 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900575 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900577 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900580 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.903412 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900583 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900585 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900589 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900591 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900594 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900596 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900599 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900601 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900604 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900608 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900611 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900614 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900617 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900619 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900622 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900624 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900627 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900629 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900631 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900634 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.903928 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900636 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900639 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900642 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900644 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900647 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900651 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900654 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900657 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900659 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900662 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900665 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900668 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900672 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900675 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900678 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900681 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900684 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900686 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900689 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.904410 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900691 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.900694 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902583 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902600 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902604 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902607 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902610 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902613 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902616 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902619 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902621 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902624 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902626 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902630 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902633 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902636 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902638 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902641 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902644 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.904897 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902647 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902650 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902653 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902656 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902658 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902661 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902664 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902666 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902669 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902671 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902674 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902676 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902679 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902681 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902689 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902692 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902697 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902701 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902704 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.905365 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902706 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902709 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902711 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902714 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902716 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902719 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902721 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902724 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902726 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902729 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902732 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902734 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902736 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902739 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902741 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902745 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902747 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902750 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902753 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902755 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.905839 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902758 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902760 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902763 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902766 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902768 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902771 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902774 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902776 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902779 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902781 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902784 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902787 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902789 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902792 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902794 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902796 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902799 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902802 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902806 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902808 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.906322 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902810 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902813 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902815 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902818 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902820 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902823 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902827 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902832 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902835 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902838 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902841 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902844 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902847 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902851 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902855 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902857 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902860 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902863 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902866 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.906826 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902869 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902872 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902875 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.902878 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902950 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902958 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902964 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902968 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902973 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902976 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902980 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902985 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902988 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902991 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902994 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.902997 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903001 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903004 2574 flags.go:64] FLAG: --cgroup-root="" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903007 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903009 2574 flags.go:64] FLAG: --client-ca-file="" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903012 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903016 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903021 2574 flags.go:64] FLAG: --cluster-domain="" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903024 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 23:37:04.907294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903027 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903030 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903034 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903038 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903041 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903044 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903047 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903050 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903053 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903056 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903059 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903062 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903066 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903069 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903072 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903075 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903077 2574 flags.go:64] FLAG: --enable-server="true" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903080 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903085 2574 flags.go:64] FLAG: --event-burst="100" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903089 2574 flags.go:64] FLAG: --event-qps="50" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903091 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903094 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903097 2574 flags.go:64] FLAG: --eviction-hard="" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903105 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 23:37:04.907887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903108 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903111 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903114 2574 flags.go:64] FLAG: --eviction-soft="" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903117 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903119 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903122 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903126 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903129 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903131 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903134 2574 flags.go:64] FLAG: --feature-gates="" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903138 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903142 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903145 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903148 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903151 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903154 2574 flags.go:64] FLAG: --help="false" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903157 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-128-69.ec2.internal" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903160 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903163 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903166 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903169 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903172 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903175 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 23:37:04.908460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903178 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903181 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903184 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903187 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903190 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903193 2574 flags.go:64] FLAG: --kube-reserved="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903196 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903199 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903202 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903204 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903207 2574 flags.go:64] FLAG: --lock-file="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903210 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903212 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903215 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903225 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903228 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903232 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903235 2574 flags.go:64] FLAG: --logging-format="text" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903237 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903241 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903243 2574 flags.go:64] FLAG: --manifest-url="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903246 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903251 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903254 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903258 2574 flags.go:64] FLAG: --max-pods="110" Jul 27 23:37:04.909059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903260 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903263 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903266 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903269 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903272 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903275 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903278 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903284 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903287 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903290 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903293 2574 flags.go:64] FLAG: --pod-cidr="" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903295 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903302 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903305 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903307 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903310 2574 flags.go:64] FLAG: --port="10250" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903313 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903316 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-000144140b23283ae" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903319 2574 flags.go:64] FLAG: --qos-reserved="" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903322 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903325 2574 flags.go:64] FLAG: --register-node="true" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903328 2574 flags.go:64] FLAG: --register-with-taints="" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903332 2574 flags.go:64] FLAG: --registry-burst="10" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903335 2574 flags.go:64] FLAG: --registry-qps="5" Jul 27 23:37:04.909683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903338 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903341 2574 flags.go:64] FLAG: --reserved-memory="" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903345 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903348 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903351 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903354 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903357 2574 flags.go:64] FLAG: --runonce="false" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903360 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903363 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903366 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903369 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903371 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903374 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903377 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903380 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903383 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903386 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903389 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903391 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903394 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903397 2574 flags.go:64] FLAG: --system-cgroups="" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903400 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903405 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903408 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903411 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 23:37:04.910297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903415 2574 flags.go:64] FLAG: --tls-min-version="" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903418 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903421 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903426 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903429 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903432 2574 flags.go:64] FLAG: --v="2" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903436 2574 flags.go:64] FLAG: --version="false" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903440 2574 flags.go:64] FLAG: --vmodule="" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903445 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.903450 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903583 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903587 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903590 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903593 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903596 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903599 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903601 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903604 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903607 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903609 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903612 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.910924 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903615 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903617 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903620 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903623 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903625 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903628 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903631 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903633 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903636 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903639 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903641 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903644 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903646 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903650 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903655 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903659 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903662 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903665 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903668 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.911436 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903671 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903676 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903679 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903681 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903684 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903687 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903689 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903692 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903695 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903697 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903700 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903702 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903705 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903707 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903710 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903712 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903715 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903717 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903720 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903723 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.911921 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903725 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903728 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903730 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903733 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903736 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903740 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903743 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903747 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903750 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903752 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903755 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903757 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903760 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903764 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903768 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903770 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903773 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903776 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903778 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903781 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.912407 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903784 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903786 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903789 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903791 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903794 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903798 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903802 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903805 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903807 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903810 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903813 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903815 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903818 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903821 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903824 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903826 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903828 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903831 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903833 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.912961 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903837 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903840 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903842 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903844 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903847 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903849 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903853 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903856 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903860 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.903862 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.904507 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.911846 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.911862 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.911882 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.911888 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 23:37:04.913441 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911945 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911950 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911954 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911957 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911960 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911963 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911966 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911969 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911972 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911975 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911978 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911980 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911983 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911985 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911988 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911991 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911993 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911996 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.911998 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912001 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.913856 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912004 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912006 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912009 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912011 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912014 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912017 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912020 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912023 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912027 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912031 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912034 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912036 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912039 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912041 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912044 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912046 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912049 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912052 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912054 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912057 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.914334 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912060 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912063 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912065 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912068 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912071 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912073 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912076 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912080 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912083 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912086 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912089 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912091 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912094 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912097 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912099 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912102 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912104 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912107 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912111 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.914855 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912113 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912116 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912121 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912124 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912127 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912130 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912133 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912136 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912139 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912141 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912144 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912146 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912149 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912153 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912156 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912159 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912162 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912165 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912168 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.915327 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912170 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912173 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912176 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912178 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912181 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912183 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912186 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912188 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912191 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912194 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912197 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912199 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912202 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912205 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912224 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912228 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912231 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912234 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912238 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912240 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.915809 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912243 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.912248 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912365 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912370 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912373 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912376 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912379 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912382 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912384 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912387 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912390 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912392 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912395 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912397 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912400 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:37:04.916338 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912402 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912405 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912408 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912410 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912413 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912416 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912419 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912421 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912424 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912427 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912429 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912438 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912441 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912444 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912447 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912449 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912451 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912454 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912456 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:37:04.916736 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912459 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912462 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912464 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912467 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912469 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912472 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912475 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912477 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912480 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912482 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912485 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912488 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912490 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912493 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912495 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912498 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912501 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912503 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912506 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912509 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:37:04.917219 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912529 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912532 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912535 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912537 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912540 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912543 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912546 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912549 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912551 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912554 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912556 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912559 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912561 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912564 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912566 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912569 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912572 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912576 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912578 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912581 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:37:04.917824 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912584 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912586 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912589 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912593 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912596 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912599 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912601 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912605 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912607 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912610 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912613 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912616 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912620 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912622 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912625 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912628 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912631 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912634 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912637 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:37:04.918301 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912640 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912643 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912645 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912648 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912651 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912653 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912656 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:04.912658 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.912664 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.912776 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.915115 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.916531 2574 server.go:1019] "Starting client certificate rotation" Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.916629 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 23:37:04.918783 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.917392 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 23:37:04.919526 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.919497 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 23:37:04.921102 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.921085 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 23:37:04.931188 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.931165 2574 log.go:25] "Validated CRI v1 runtime API" Jul 27 23:37:04.933630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.933616 2574 log.go:25] "Validated CRI v1 image API" Jul 27 23:37:04.937462 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.937445 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 23:37:04.941477 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.941458 2574 fs.go:135] Filesystem UUIDs: map[35351d28-9377-4f49-a5bc-494b5751ba7e:/dev/nvme0n1p3 4651845d-c057-4209-bf33-b7fdfe30bd33:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 27 23:37:04.941541 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.941478 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 23:37:04.945916 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.945805 2574 manager.go:216] Machine: {Timestamp:2026-07-27 23:37:04.945307748 +0000 UTC m=+0.305844423 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093386 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec282fed00ac32afc15448fd141bf7eb SystemUUID:ec282fed-00ac-32af-c154-48fd141bf7eb BootID:89e26241-1e1a-49a1-9d30-a7c77363fad7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:95:c5:18:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:95:c5:18:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:33:9c:94:74:b4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 23:37:04.945916 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.945914 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 23:37:04.946024 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.945993 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 23:37:04.947509 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.947494 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 23:37:04.949005 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.948982 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 23:37:04.949141 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.949009 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-69.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 23:37:04.949752 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.949742 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 23:37:04.949784 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.949754 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 23:37:04.949784 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.949761 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 23:37:04.950401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.950391 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 23:37:04.951613 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.951603 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 23:37:04.953194 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953183 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 27 23:37:04.953305 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953296 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 23:37:04.953337 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953324 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 27 23:37:04.953337 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953335 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 23:37:04.953401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953350 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 23:37:04.953401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953358 2574 kubelet.go:398] "Adding apiserver pod source" Jul 27 23:37:04.953401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953366 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 23:37:04.953746 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953731 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 23:37:04.953821 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.953768 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 23:37:04.956193 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.956178 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 23:37:04.958357 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.958344 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 23:37:04.958396 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.958374 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 23:37:04.960161 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960150 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 23:37:04.960203 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960168 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 23:37:04.960203 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960174 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 23:37:04.960203 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960183 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 23:37:04.960203 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960197 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960205 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960215 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960220 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960227 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960234 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960248 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 23:37:04.960308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960254 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 23:37:04.960952 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960941 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 23:37:04.960987 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.960954 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 23:37:04.963238 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.963220 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-69.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 23:37:04.963238 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.963226 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-69.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 23:37:04.963318 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.963234 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 23:37:04.964510 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.964499 2574 server.go:1268] "Started kubelet" Jul 27 23:37:04.964623 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.964597 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 23:37:04.965048 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.964994 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 23:37:04.965130 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.965073 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 23:37:04.965390 ip-10-0-128-69 systemd[1]: Started Kubernetes Kubelet. Jul 27 23:37:04.966637 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.966624 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 27 23:37:04.969362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.968507 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 23:37:04.971315 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.971292 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 23:37:04.974902 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.974886 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mbz9b" Jul 27 23:37:04.976559 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.975707 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-69.ec2.internal.18c649c7d7a8ed8d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-69.ec2.internal,UID:ip-10-0-128-69.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-69.ec2.internal,},FirstTimestamp:2026-07-27 23:37:04.964472205 +0000 UTC m=+0.325008874,LastTimestamp:2026-07-27 23:37:04.964472205 +0000 UTC m=+0.325008874,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-69.ec2.internal,}" Jul 27 23:37:04.978199 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.978176 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 23:37:04.978278 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.978178 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 23:37:04.978339 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.978294 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:04.978943 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.978926 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 23:37:04.978943 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.978944 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 23:37:04.979088 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.978929 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 23:37:04.979088 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979030 2574 factory.go:55] Registering systemd factory Jul 27 23:37:04.979088 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979047 2574 factory.go:223] Registration of the systemd container factory successfully Jul 27 23:37:04.979218 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979092 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 27 23:37:04.979218 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979102 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 27 23:37:04.979319 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979305 2574 factory.go:153] Registering CRI-O factory Jul 27 23:37:04.979358 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979322 2574 factory.go:223] Registration of the crio container factory successfully Jul 27 23:37:04.979395 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979371 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 23:37:04.979445 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979395 2574 factory.go:103] Registering Raw factory Jul 27 23:37:04.979445 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979410 2574 manager.go:1196] Started watching for new ooms in manager Jul 27 23:37:04.979920 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.979898 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mbz9b" Jul 27 23:37:04.980107 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.980091 2574 manager.go:349] Starting recovery of all containers Jul 27 23:37:04.989606 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.989580 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:04.989897 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.989882 2574 manager.go:354] Recovery completed Jul 27 23:37:04.992052 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:04.992020 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-69.ec2.internal\" not found" node="ip-10-0-128-69.ec2.internal" Jul 27 23:37:04.995256 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.995244 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:04.997240 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997217 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:04.997307 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997258 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:04.997307 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997271 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:04.997841 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997824 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 23:37:04.997841 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997834 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:04.997841 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997838 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 23:37:04.998026 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.997857 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 27 23:37:04.998706 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.998681 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:04.998782 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.998720 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:04.998782 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.998743 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:04.999721 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.999678 2574 policy_none.go:49] "None policy: Start" Jul 27 23:37:04.999792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.999732 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 23:37:04.999792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:04.999756 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 23:37:05.000977 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.000957 2574 policy_none.go:47] "Start" Jul 27 23:37:05.039304 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039137 2574 manager.go:337] "Starting Device Plugin manager" Jul 27 23:37:05.039409 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.039331 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 23:37:05.039409 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039345 2574 server.go:87] "Starting device plugin registration server" Jul 27 23:37:05.039612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039599 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 23:37:05.039678 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039616 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 23:37:05.039753 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039732 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 23:37:05.039861 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039847 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 23:37:05.039920 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.039864 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 23:37:05.040395 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.040369 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 23:37:05.040473 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.040432 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.140751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.140719 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:05.142839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.141792 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:05.142839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.141820 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:05.142839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.141830 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:05.142839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.141859 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.149617 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.149560 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.149617 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.149589 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-69.ec2.internal\": node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.165229 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.165199 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 23:37:05.166332 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.166311 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 23:37:05.166434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.166343 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 23:37:05.166434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.166366 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 23:37:05.166434 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.166395 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 27 23:37:05.168695 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.168673 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.168789 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.168725 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:05.268464 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.268418 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal"] Jul 27 23:37:05.268638 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.268572 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:05.268774 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.268753 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.270445 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.270427 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:05.270536 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.270463 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:05.270536 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.270479 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:05.272879 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.272866 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:05.273031 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273014 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.273071 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273055 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:05.273857 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273839 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:05.273857 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273852 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:05.273976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273873 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:05.273976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273888 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:05.273976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273875 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:05.273976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.273959 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:05.276277 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.276261 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.276333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.276287 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:37:05.277034 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.277007 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:37:05.277034 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.277034 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:37:05.277191 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.277048 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:37:05.297247 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.297226 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-69.ec2.internal\" not found" node="ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.301977 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.301958 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-69.ec2.internal\" not found" node="ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.369904 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.369878 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.380218 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.380193 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a62480b743c193092f01b29ef1c2800-config\") pod \"kube-apiserver-proxy-ip-10-0-128-69.ec2.internal\" (UID: \"6a62480b743c193092f01b29ef1c2800\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.380288 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.380222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.380288 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.380244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.470547 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.470434 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.480503 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a62480b743c193092f01b29ef1c2800-config\") pod \"kube-apiserver-proxy-ip-10-0-128-69.ec2.internal\" (UID: \"6a62480b743c193092f01b29ef1c2800\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.480580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.480580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.480580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480562 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a62480b743c193092f01b29ef1c2800-config\") pod \"kube-apiserver-proxy-ip-10-0-128-69.ec2.internal\" (UID: \"6a62480b743c193092f01b29ef1c2800\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.480682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480600 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.480682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.480621 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/877cdd6d63a4542a63dc44588a323269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal\" (UID: \"877cdd6d63a4542a63dc44588a323269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.578048 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.577998 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.601079 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.601048 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.606848 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.606617 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:05.681651 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.681600 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.784441 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.784357 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.790780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.790756 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:05.885645 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.885603 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.916958 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.916916 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 23:37:05.917475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.917075 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 23:37:05.917475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.917112 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 23:37:05.917475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.917126 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 23:37:05.986402 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:05.986370 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:05.987580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.987551 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 23:32:05 +0000 UTC" deadline="2028-05-06 22:37:02.502896432 +0000 UTC" Jul 27 23:37:05.987646 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:05.987582 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15574h59m56.515318246s" Jul 27 23:37:06.086757 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.086676 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:06.187095 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.187058 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:06.221609 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:06.221569 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877cdd6d63a4542a63dc44588a323269.slice/crio-7cf61870afa824f6b044f142d34f83fcefab4a0b22657e27231769936a64d12b WatchSource:0}: Error finding container 7cf61870afa824f6b044f142d34f83fcefab4a0b22657e27231769936a64d12b: Status 404 returned error can't find the container with id 7cf61870afa824f6b044f142d34f83fcefab4a0b22657e27231769936a64d12b Jul 27 23:37:06.228432 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.228350 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:37:06.233721 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:06.233688 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a62480b743c193092f01b29ef1c2800.slice/crio-e228d0bb484e63988fd4a5586178200de08cc5b6eb142c69f3e7daf0f3e8d7da WatchSource:0}: Error finding container e228d0bb484e63988fd4a5586178200de08cc5b6eb142c69f3e7daf0f3e8d7da: Status 404 returned error can't find the container with id e228d0bb484e63988fd4a5586178200de08cc5b6eb142c69f3e7daf0f3e8d7da Jul 27 23:37:06.287582 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.287545 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-69.ec2.internal\" not found" Jul 27 23:37:06.321609 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.321583 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:06.380210 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.380114 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" Jul 27 23:37:06.391277 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.391238 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 23:37:06.392052 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.392037 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" Jul 27 23:37:06.398305 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.398287 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 23:37:06.956253 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.956214 2574 apiserver.go:52] "Watching apiserver" Jul 27 23:37:06.964780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.964754 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 23:37:06.966344 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.965894 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rm6tp","kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8","openshift-cluster-node-tuning-operator/tuned-hsc4t","openshift-dns/node-resolver-xd4ck","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal","openshift-multus/multus-additional-cni-plugins-dzh5j","openshift-image-registry/node-ca-rhcb4","openshift-multus/multus-kvhtj","openshift-multus/network-metrics-daemon-4w58m","openshift-network-diagnostics/network-check-target-nr6fd","openshift-network-operator/iptables-alerter-sj786","openshift-ovn-kubernetes/ovnkube-node-ksj56","kube-system/global-pull-secret-syncer-x7j27"] Jul 27 23:37:06.968690 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.968628 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.971704 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971431 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.971704 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971493 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 23:37:06.971704 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971580 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.971704 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971493 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.971704 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971586 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 23:37:06.972021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.971818 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fbqzt\"" Jul 27 23:37:06.972083 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.972032 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 23:37:06.974617 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.974061 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.975297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.974875 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 23:37:06.975297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.975063 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rf8cr\"" Jul 27 23:37:06.975297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.975127 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.975297 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.975274 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 23:37:06.975596 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.975324 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.975596 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.975449 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 23:37:06.976269 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.976245 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.976386 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.976367 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 23:37:06.976665 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.976647 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-47kzq\"" Jul 27 23:37:06.976752 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.976663 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.976819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.976749 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 23:37:06.977151 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.977132 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.979414 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.979173 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nt2r8\"" Jul 27 23:37:06.979414 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.979262 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:06.979414 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.979385 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.979644 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.979532 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 23:37:06.979722 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.979704 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.981721 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.981688 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-74dl6\"" Jul 27 23:37:06.982022 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.982003 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.982283 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.982262 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.982497 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.982478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:06.982600 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.982557 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:06.984725 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.984702 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvhtj" Jul 27 23:37:06.987800 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.987103 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 23:37:06.987800 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.987337 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vt7nb\"" Jul 27 23:37:06.987800 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.987438 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:06.989752 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.989735 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 23:37:06.990123 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990105 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 23:37:06.990210 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990190 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c76qs\"" Jul 27 23:37:06.990534 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.990639 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-dbus\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:06.990639 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.990639 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-log-socket\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovn-node-metrics-cert\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-tmp\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a22761-c48a-40c9-a7b7-5206184e6615-tmp-dir\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990715 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-var-lib-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-netd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990764 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-script-lib\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.990808 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990788 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jdbm\" (UniqueName: \"kubernetes.io/projected/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-kube-api-access-4jdbm\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-device-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990861 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-run\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990112 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990932 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-sys\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.990997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysconfig\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991028 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-kubernetes\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a22761-c48a-40c9-a7b7-5206184e6615-hosts-file\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991174 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-ovn\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991226 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991211 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991239 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-kubelet\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-host\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-netns\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991513 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-socket-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991544 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-sys-fs\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991560 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-lib-modules\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991618 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-slash\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-etc-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-env-overrides\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991719 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-var-lib-kubelet\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.991751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991757 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-os-release\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.992454 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.991784 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6qfq\" (UniqueName: \"kubernetes.io/projected/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-kube-api-access-n6qfq\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.993160 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993137 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:06.993272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993159 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-config\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.993272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjxn9\" (UniqueName: \"kubernetes.io/projected/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-kube-api-access-xjxn9\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-node-log\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cnibin\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993764 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.993843 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-bin\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993906 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-registration-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.993946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-conf\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994130 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svpdl\" (UniqueName: \"kubernetes.io/projected/25a22761-c48a-40c9-a7b7-5206184e6615-kube-api-access-svpdl\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-systemd-units\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.994642 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-tuned\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994660 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8d749\"" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-system-cni-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994784 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-modprobe-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-systemd\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994882 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-kubelet-config\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-systemd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994941 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.994964 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjhp\" (UniqueName: \"kubernetes.io/projected/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kube-api-access-gxjhp\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:06.995460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.995267 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 23:37:06.997633 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:06.997497 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:06.997633 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:06.997594 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:07.001065 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.000837 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.003208 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.003108 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:07.003306 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.003277 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 23:37:07.003375 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.003156 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:07.003431 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.003322 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z5ndr\"" Jul 27 23:37:07.010948 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.010932 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g75j5" Jul 27 23:37:07.018556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.018342 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g75j5" Jul 27 23:37:07.080583 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.080557 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 23:37:07.096176 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096134 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-node-log\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cnibin\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096209 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-node-log\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-kubelet\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-multus-certs\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096282 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cnibin\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096292 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6abb1cb5-2321-4511-84a8-489581835a9a-host-slash\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-bin\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096354 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-registration-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-conf\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096438 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-registration-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096455 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-bin\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svpdl\" (UniqueName: \"kubernetes.io/projected/25a22761-c48a-40c9-a7b7-5206184e6615-kube-api-access-svpdl\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096528 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-conf\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-socket-dir-parent\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-netns\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096635 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-konnectivity-ca\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-systemd-units\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096719 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096744 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-tuned\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-system-cni-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.096802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-bin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096838 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-system-cni-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096832 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-systemd-units\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-modprobe-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096907 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096918 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-systemd\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096802 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-kubelet-config\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.096980 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097011 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-cni-binary-copy\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-multus\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097053 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-modprobe-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097058 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9fzb\" (UniqueName: \"kubernetes.io/projected/6abb1cb5-2321-4511-84a8-489581835a9a-kube-api-access-c9fzb\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097067 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-systemd\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-systemd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097125 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjhp\" (UniqueName: \"kubernetes.io/projected/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kube-api-access-gxjhp\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.097481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-kubelet-config\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097153 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnp4w\" (UniqueName: \"kubernetes.io/projected/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-kube-api-access-rnp4w\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097172 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-systemd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-dbus\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097205 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-daemon-config\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-log-socket\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovn-node-metrics-cert\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-tmp\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c884103-50ba-403b-89eb-66d666438a5c-dbus\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a22761-c48a-40c9-a7b7-5206184e6615-tmp-dir\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097438 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-conf-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqdc4\" (UniqueName: \"kubernetes.io/projected/66c4f215-ea76-4982-a115-bc31e5efcd77-kube-api-access-fqdc4\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-var-lib-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097530 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-netd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097528 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysctl-d\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.098621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-script-lib\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jdbm\" (UniqueName: \"kubernetes.io/projected/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-kube-api-access-4jdbm\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-device-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-run\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-sys\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6abb1cb5-2321-4511-84a8-489581835a9a-iptables-alerter-script\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysconfig\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097793 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-cnibin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097821 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097849 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-kubernetes\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a22761-c48a-40c9-a7b7-5206184e6615-hosts-file\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097954 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-os-release\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097980 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-ovn\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098052 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.099806 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098191 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-script-lib\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097466 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-log-socket\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098290 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-var-lib-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098338 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-cni-netd\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098339 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-device-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.097818 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a22761-c48a-40c9-a7b7-5206184e6615-tmp-dir\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098383 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-run\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098436 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-sys\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098445 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-kubernetes\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a22761-c48a-40c9-a7b7-5206184e6615-hosts-file\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098497 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098054 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098529 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098504 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-sysconfig\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098547 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-run-ovn\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098610 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.098639 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:07.100819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-kubelet\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098686 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-kubelet\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.098734 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:07.598693677 +0000 UTC m=+2.959230362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098764 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-host\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098795 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098825 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-etc-kubernetes\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098849 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-host\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-netns\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-socket-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-sys-fs\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098985 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-lib-modules\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099009 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-system-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099014 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099035 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-k8s-cni-cncf-io\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099080 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-agent-certs\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099107 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-slash\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.102241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099127 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-socket-dir\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099134 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-etc-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.098850 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-host\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-env-overrides\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-etc-openvswitch\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-var-lib-kubelet\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099227 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-run-netns\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099157 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-os-release\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099260 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-host-slash\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qfq\" (UniqueName: \"kubernetes.io/projected/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-kube-api-access-n6qfq\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099333 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-hostroot\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-serviceca\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099403 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-os-release\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099463 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-lib-modules\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099492 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-sys-fs\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099514 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-var-lib-kubelet\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.103415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099552 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-config\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099573 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-env-overrides\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjxn9\" (UniqueName: \"kubernetes.io/projected/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-kube-api-access-xjxn9\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099731 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57b84b79-a6f8-4c85-ba66-4bd9a964961a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.099848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk9fz\" (UniqueName: \"kubernetes.io/projected/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-kube-api-access-pk9fz\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.100073 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovnkube-config\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.101045 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-tmp\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.101106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-etc-tuned\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.104440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.101306 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-ovn-node-metrics-cert\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.109060 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.109031 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svpdl\" (UniqueName: \"kubernetes.io/projected/25a22761-c48a-40c9-a7b7-5206184e6615-kube-api-access-svpdl\") pod \"node-resolver-xd4ck\" (UID: \"25a22761-c48a-40c9-a7b7-5206184e6615\") " pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.112482 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.112430 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjhp\" (UniqueName: \"kubernetes.io/projected/57b84b79-a6f8-4c85-ba66-4bd9a964961a-kube-api-access-gxjhp\") pod \"aws-ebs-csi-driver-node-w5vd8\" (UID: \"57b84b79-a6f8-4c85-ba66-4bd9a964961a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.114515 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.114487 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjxn9\" (UniqueName: \"kubernetes.io/projected/8ef8a9b6-fcb1-4448-9fc6-d728dda2f952-kube-api-access-xjxn9\") pod \"tuned-hsc4t\" (UID: \"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952\") " pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.114730 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.114710 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qfq\" (UniqueName: \"kubernetes.io/projected/3b755624-4b79-4d54-bfc2-6ac2fd632c5f-kube-api-access-n6qfq\") pod \"multus-additional-cni-plugins-dzh5j\" (UID: \"3b755624-4b79-4d54-bfc2-6ac2fd632c5f\") " pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.116164 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.116122 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jdbm\" (UniqueName: \"kubernetes.io/projected/5cb08f9e-fb93-44ff-aee9-aa4e38b10d44-kube-api-access-4jdbm\") pod \"ovnkube-node-ksj56\" (UID: \"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.145030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.144997 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:07.171119 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.171058 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" event={"ID":"877cdd6d63a4542a63dc44588a323269","Type":"ContainerStarted","Data":"7cf61870afa824f6b044f142d34f83fcefab4a0b22657e27231769936a64d12b"} Jul 27 23:37:07.172329 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.172298 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" event={"ID":"6a62480b743c193092f01b29ef1c2800","Type":"ContainerStarted","Data":"e228d0bb484e63988fd4a5586178200de08cc5b6eb142c69f3e7daf0f3e8d7da"} Jul 27 23:37:07.200219 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6abb1cb5-2321-4511-84a8-489581835a9a-iptables-alerter-script\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-cnibin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200264 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-os-release\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-etc-kubernetes\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-host\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200375 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-cnibin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-system-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200428 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-os-release\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200449 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-k8s-cni-cncf-io\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200460 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-etc-kubernetes\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200433 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-system-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-agent-certs\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200515 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-k8s-cni-cncf-io\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200526 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-hostroot\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200557 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-hostroot\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-serviceca\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk9fz\" (UniqueName: \"kubernetes.io/projected/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-kube-api-access-pk9fz\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200655 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-kubelet\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200680 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-multus-certs\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200705 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6abb1cb5-2321-4511-84a8-489581835a9a-host-slash\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-socket-dir-parent\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200742 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-host\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-netns\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.200792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-konnectivity-ca\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-bin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200882 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200910 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-cni-binary-copy\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-multus\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6abb1cb5-2321-4511-84a8-489581835a9a-iptables-alerter-script\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9fzb\" (UniqueName: \"kubernetes.io/projected/6abb1cb5-2321-4511-84a8-489581835a9a-kube-api-access-c9fzb\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200998 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnp4w\" (UniqueName: \"kubernetes.io/projected/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-kube-api-access-rnp4w\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.200998 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-netns\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201049 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-serviceca\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201090 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-kubelet\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201112 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-bin\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201160 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6abb1cb5-2321-4511-84a8-489581835a9a-host-slash\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201177 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-cni-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201217 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-var-lib-cni-multus\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-socket-dir-parent\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201305 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-host-run-multus-certs\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.201535 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.201385 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.201454 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:07.701434933 +0000 UTC m=+3.061971600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201384 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-daemon-config\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-conf-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201582 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqdc4\" (UniqueName: \"kubernetes.io/projected/66c4f215-ea76-4982-a115-bc31e5efcd77-kube-api-access-fqdc4\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201624 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-cni-binary-copy\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201688 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-conf-dir\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.201954 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66c4f215-ea76-4982-a115-bc31e5efcd77-multus-daemon-config\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.202488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.202252 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-konnectivity-ca\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.204475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.204446 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd1e47f9-2f7a-4523-92f3-437b99a9f63a-agent-certs\") pod \"konnectivity-agent-rm6tp\" (UID: \"bd1e47f9-2f7a-4523-92f3-437b99a9f63a\") " pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.208590 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.208424 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:07.208590 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.208450 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:07.208590 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.208466 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:07.208590 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.208554 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:07.708518987 +0000 UTC m=+3.069055667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:07.211274 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.211229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9fzb\" (UniqueName: \"kubernetes.io/projected/6abb1cb5-2321-4511-84a8-489581835a9a-kube-api-access-c9fzb\") pod \"iptables-alerter-sj786\" (UID: \"6abb1cb5-2321-4511-84a8-489581835a9a\") " pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.211683 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.211662 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk9fz\" (UniqueName: \"kubernetes.io/projected/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-kube-api-access-pk9fz\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.211797 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.211776 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnp4w\" (UniqueName: \"kubernetes.io/projected/6e6dfd4f-793c-477c-8bde-43d14d6e1ac6-kube-api-access-rnp4w\") pod \"node-ca-rhcb4\" (UID: \"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6\") " pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.211849 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.211811 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqdc4\" (UniqueName: \"kubernetes.io/projected/66c4f215-ea76-4982-a115-bc31e5efcd77-kube-api-access-fqdc4\") pod \"multus-kvhtj\" (UID: \"66c4f215-ea76-4982-a115-bc31e5efcd77\") " pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.282218 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.282176 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" Jul 27 23:37:07.291624 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.291585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:07.304914 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.304891 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" Jul 27 23:37:07.312024 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.311993 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" Jul 27 23:37:07.322272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.322249 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xd4ck" Jul 27 23:37:07.330372 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.330346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvhtj" Jul 27 23:37:07.339480 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.339459 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:07.346675 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.346656 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rhcb4" Jul 27 23:37:07.348199 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.348179 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:07.355474 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.355455 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sj786" Jul 27 23:37:07.605716 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.605619 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:07.605904 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.605825 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:07.605972 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.605915 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:08.605894358 +0000 UTC m=+3.966431023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:07.706797 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.706758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:07.706960 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.706914 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:07.707008 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.706976 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:08.706961071 +0000 UTC m=+4.067497728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:07.807984 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.807946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:07.808211 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.808176 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:07.808211 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.808217 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:07.808402 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.808234 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:07.808402 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:07.808313 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:08.808289506 +0000 UTC m=+4.168826177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:07.843869 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:07.843834 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:37:08.022829 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.022744 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 23:32:07 +0000 UTC" deadline="2028-01-12 21:10:18.569866857 +0000 UTC" Jul 27 23:37:08.022829 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.022777 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12813h33m10.547093634s" Jul 27 23:37:08.166959 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.166925 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:08.167162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.166927 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:08.167162 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.167047 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:08.167162 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.167151 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:08.292965 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.292923 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abb1cb5_2321_4511_84a8_489581835a9a.slice/crio-2e8a00fcd642ac6a4f6a5ba7e81b06a0b1101b1f33482fcf7b6f5fc4066932c8 WatchSource:0}: Error finding container 2e8a00fcd642ac6a4f6a5ba7e81b06a0b1101b1f33482fcf7b6f5fc4066932c8: Status 404 returned error can't find the container with id 2e8a00fcd642ac6a4f6a5ba7e81b06a0b1101b1f33482fcf7b6f5fc4066932c8 Jul 27 23:37:08.294140 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.294119 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef8a9b6_fcb1_4448_9fc6_d728dda2f952.slice/crio-91037622a74522b995bbef584701f796d97108d87ff87230f0dd83f4bb242c17 WatchSource:0}: Error finding container 91037622a74522b995bbef584701f796d97108d87ff87230f0dd83f4bb242c17: Status 404 returned error can't find the container with id 91037622a74522b995bbef584701f796d97108d87ff87230f0dd83f4bb242c17 Jul 27 23:37:08.295764 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.295685 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb08f9e_fb93_44ff_aee9_aa4e38b10d44.slice/crio-8101db0a0ff8f9df82d9eee96864228a4e5f472266892099f52ee78c2745f3dd WatchSource:0}: Error finding container 8101db0a0ff8f9df82d9eee96864228a4e5f472266892099f52ee78c2745f3dd: Status 404 returned error can't find the container with id 8101db0a0ff8f9df82d9eee96864228a4e5f472266892099f52ee78c2745f3dd Jul 27 23:37:08.295974 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.295944 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a22761_c48a_40c9_a7b7_5206184e6615.slice/crio-c121dee95779a375778d6a8469f2f539a6f6aba4ebf803564f95a1b630b4c83d WatchSource:0}: Error finding container c121dee95779a375778d6a8469f2f539a6f6aba4ebf803564f95a1b630b4c83d: Status 404 returned error can't find the container with id c121dee95779a375778d6a8469f2f539a6f6aba4ebf803564f95a1b630b4c83d Jul 27 23:37:08.298203 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.298141 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c4f215_ea76_4982_a115_bc31e5efcd77.slice/crio-a8990818eec4330c784bdc324b7e00b63af49dbebbb344c877b18dcb04d85e9e WatchSource:0}: Error finding container a8990818eec4330c784bdc324b7e00b63af49dbebbb344c877b18dcb04d85e9e: Status 404 returned error can't find the container with id a8990818eec4330c784bdc324b7e00b63af49dbebbb344c877b18dcb04d85e9e Jul 27 23:37:08.300371 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.300349 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b755624_4b79_4d54_bfc2_6ac2fd632c5f.slice/crio-2bb74e8e86278f93e96c607f7ec9749deb03374ae48ce9745756e63ad4bc77b5 WatchSource:0}: Error finding container 2bb74e8e86278f93e96c607f7ec9749deb03374ae48ce9745756e63ad4bc77b5: Status 404 returned error can't find the container with id 2bb74e8e86278f93e96c607f7ec9749deb03374ae48ce9745756e63ad4bc77b5 Jul 27 23:37:08.301463 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.301437 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1e47f9_2f7a_4523_92f3_437b99a9f63a.slice/crio-6525c650b5efa17bf04f1ae371ef6e701c1e7eccc0d5a51e7b60c1188c4e1fd8 WatchSource:0}: Error finding container 6525c650b5efa17bf04f1ae371ef6e701c1e7eccc0d5a51e7b60c1188c4e1fd8: Status 404 returned error can't find the container with id 6525c650b5efa17bf04f1ae371ef6e701c1e7eccc0d5a51e7b60c1188c4e1fd8 Jul 27 23:37:08.302101 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.302074 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6dfd4f_793c_477c_8bde_43d14d6e1ac6.slice/crio-1fc23b641e4cb57e62dfccf50ee96d2e5b930744726c8a5e6d7d2ed87369f833 WatchSource:0}: Error finding container 1fc23b641e4cb57e62dfccf50ee96d2e5b930744726c8a5e6d7d2ed87369f833: Status 404 returned error can't find the container with id 1fc23b641e4cb57e62dfccf50ee96d2e5b930744726c8a5e6d7d2ed87369f833 Jul 27 23:37:08.303333 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:08.303218 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b84b79_a6f8_4c85_ba66_4bd9a964961a.slice/crio-5010ba54669dffe415774d2e37412a1690cdb68c953cb37de07341575caa3aef WatchSource:0}: Error finding container 5010ba54669dffe415774d2e37412a1690cdb68c953cb37de07341575caa3aef: Status 404 returned error can't find the container with id 5010ba54669dffe415774d2e37412a1690cdb68c953cb37de07341575caa3aef Jul 27 23:37:08.614467 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.614422 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:08.614681 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.614607 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:08.614725 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.614703 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:10.614681302 +0000 UTC m=+5.975217960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:08.715684 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.715641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:08.715886 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.715832 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:08.715949 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.715901 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:10.715880125 +0000 UTC m=+6.076416800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:08.817353 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:08.816601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:08.817353 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.816824 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:08.817353 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.816848 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:08.817353 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.816863 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:08.817353 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:08.816931 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:10.816909318 +0000 UTC m=+6.177445993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:09.023149 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.023077 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 23:32:07 +0000 UTC" deadline="2028-01-29 06:58:38.461725828 +0000 UTC" Jul 27 23:37:09.023149 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.023146 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13207h21m29.438584292s" Jul 27 23:37:09.170463 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.169904 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:09.170463 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:09.170045 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:09.185662 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.184938 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" event={"ID":"6a62480b743c193092f01b29ef1c2800","Type":"ContainerStarted","Data":"9cd657c5b5cde77a4b8965c5b337dd7fcb7bee6e462bd0cd1b5d3791c199b858"} Jul 27 23:37:09.187911 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.187881 2574 generic.go:356] "Generic (PLEG): container finished" podID="877cdd6d63a4542a63dc44588a323269" containerID="158d8a45ca4aee4bffd852b1c0bf0dd560e9c863fa817d4b55769af175d5d435" exitCode=0 Jul 27 23:37:09.188043 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.187966 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" event={"ID":"877cdd6d63a4542a63dc44588a323269","Type":"ContainerDied","Data":"158d8a45ca4aee4bffd852b1c0bf0dd560e9c863fa817d4b55769af175d5d435"} Jul 27 23:37:09.193551 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.193512 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" event={"ID":"57b84b79-a6f8-4c85-ba66-4bd9a964961a","Type":"ContainerStarted","Data":"5010ba54669dffe415774d2e37412a1690cdb68c953cb37de07341575caa3aef"} Jul 27 23:37:09.198241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.198186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rm6tp" event={"ID":"bd1e47f9-2f7a-4523-92f3-437b99a9f63a","Type":"ContainerStarted","Data":"6525c650b5efa17bf04f1ae371ef6e701c1e7eccc0d5a51e7b60c1188c4e1fd8"} Jul 27 23:37:09.199536 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.199475 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-69.ec2.internal" podStartSLOduration=3.199456918 podStartE2EDuration="3.199456918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:09.199035024 +0000 UTC m=+4.559571742" watchObservedRunningTime="2026-07-27 23:37:09.199456918 +0000 UTC m=+4.559993603" Jul 27 23:37:09.208435 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.208406 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerStarted","Data":"2bb74e8e86278f93e96c607f7ec9749deb03374ae48ce9745756e63ad4bc77b5"} Jul 27 23:37:09.225607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.225545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xd4ck" event={"ID":"25a22761-c48a-40c9-a7b7-5206184e6615","Type":"ContainerStarted","Data":"c121dee95779a375778d6a8469f2f539a6f6aba4ebf803564f95a1b630b4c83d"} Jul 27 23:37:09.234455 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.234315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"8101db0a0ff8f9df82d9eee96864228a4e5f472266892099f52ee78c2745f3dd"} Jul 27 23:37:09.241940 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.241855 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sj786" event={"ID":"6abb1cb5-2321-4511-84a8-489581835a9a","Type":"ContainerStarted","Data":"2e8a00fcd642ac6a4f6a5ba7e81b06a0b1101b1f33482fcf7b6f5fc4066932c8"} Jul 27 23:37:09.254594 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.254486 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rhcb4" event={"ID":"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6","Type":"ContainerStarted","Data":"1fc23b641e4cb57e62dfccf50ee96d2e5b930744726c8a5e6d7d2ed87369f833"} Jul 27 23:37:09.269609 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.269563 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvhtj" event={"ID":"66c4f215-ea76-4982-a115-bc31e5efcd77","Type":"ContainerStarted","Data":"a8990818eec4330c784bdc324b7e00b63af49dbebbb344c877b18dcb04d85e9e"} Jul 27 23:37:09.275336 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:09.275274 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" event={"ID":"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952","Type":"ContainerStarted","Data":"91037622a74522b995bbef584701f796d97108d87ff87230f0dd83f4bb242c17"} Jul 27 23:37:10.168097 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.167251 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:10.168097 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.167395 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:10.168097 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.167880 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:10.168097 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.168039 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:10.292890 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.292846 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" event={"ID":"877cdd6d63a4542a63dc44588a323269","Type":"ContainerStarted","Data":"fa5c9633d2ca6bfde0afb27555cfaa73ad003be474cf365ab1d45f7f60cf762b"} Jul 27 23:37:10.629075 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.628978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:10.629291 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.629204 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:10.629291 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.629274 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.629254643 +0000 UTC m=+9.989791317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:10.731148 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.731106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:10.731347 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.731308 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:10.731406 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.731378 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.731356774 +0000 UTC m=+10.091893434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:10.832406 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:10.832359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:10.832634 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.832566 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:10.832634 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.832589 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:10.832634 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.832605 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:10.832805 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:10.832672 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.832650776 +0000 UTC m=+10.193187439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:11.170075 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:11.169495 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:11.170075 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:11.169663 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:12.166893 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:12.166853 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:12.167084 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:12.166993 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:12.167432 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:12.167411 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:12.167601 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:12.167538 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:13.170789 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:13.170295 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:13.170789 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:13.170429 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:14.167369 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:14.166743 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:14.167369 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.166857 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:14.167369 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:14.167206 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:14.167369 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.167294 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:14.668427 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:14.668387 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:14.668896 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.668612 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:14.668896 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.668680 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:22.668661382 +0000 UTC m=+18.029198055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:14.769913 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:14.769783 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:14.770166 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.769980 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:14.770166 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.770092 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:22.770068287 +0000 UTC m=+18.130604948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:14.871233 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:14.871195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:14.871484 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.871460 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:14.871571 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.871529 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:14.871571 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.871546 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:14.871671 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:14.871622 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:22.87158942 +0000 UTC m=+18.232126102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:15.168594 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:15.167996 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:15.168594 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:15.168122 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:16.167951 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:16.167300 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:16.167951 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:16.167428 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:16.167951 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:16.167508 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:16.167951 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:16.167600 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:17.168219 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:17.167717 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:17.168219 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:17.167863 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:18.166926 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:18.166892 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:18.167105 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:18.166892 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:18.167105 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:18.167006 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:18.167231 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:18.167104 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:19.167650 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:19.167612 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:19.168078 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:19.167767 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:20.166709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:20.166661 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:20.166709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:20.166692 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:20.166907 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:20.166796 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:20.166997 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:20.166918 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:21.167363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:21.167327 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:21.167807 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:21.167474 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:22.167388 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:22.167352 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:22.167923 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:22.167353 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:22.167923 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.167484 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:22.167923 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.167560 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:22.731873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:22.731842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:22.732110 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.731978 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:22.732110 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.732033 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:38.73201873 +0000 UTC m=+34.092555390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:22.832481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:22.832454 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:22.832618 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.832571 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:22.832657 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.832622 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:38.832605531 +0000 UTC m=+34.193142192 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:22.933368 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:22.933333 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:22.933536 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.933471 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:22.933536 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.933485 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:22.933536 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.933495 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:22.933646 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:22.933557 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:37:38.9335405 +0000 UTC m=+34.294077159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:23.167177 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:23.167141 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:23.167350 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:23.167274 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:24.166600 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:24.166569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:24.167036 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:24.166569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:24.167036 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:24.166694 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:24.167036 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:24.166784 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:25.167958 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:25.167919 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:25.168430 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:25.168051 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:26.166872 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:26.166835 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:26.167120 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:26.166836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:26.167120 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:26.166963 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:26.167120 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:26.167039 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:27.169901 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.169391 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:27.169901 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:27.169534 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:27.328991 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.328959 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rm6tp" event={"ID":"bd1e47f9-2f7a-4523-92f3-437b99a9f63a","Type":"ContainerStarted","Data":"89e9086b006dabe998788ed4e22a1cdfeba890474f09f97fa8b24529199eea56"} Jul 27 23:37:27.331386 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.331339 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="75a2a5124d93c9343bbf75fe0e45f3e690c5d2ce167c37d8e823f8e9749fa1e3" exitCode=0 Jul 27 23:37:27.331463 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.331404 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"75a2a5124d93c9343bbf75fe0e45f3e690c5d2ce167c37d8e823f8e9749fa1e3"} Jul 27 23:37:27.333040 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.333017 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xd4ck" event={"ID":"25a22761-c48a-40c9-a7b7-5206184e6615","Type":"ContainerStarted","Data":"c3f1408a2187912d9098505f1a376628fabeefd6e1885fb720bc9f475e5ea513"} Jul 27 23:37:27.334645 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.334622 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rhcb4" event={"ID":"6e6dfd4f-793c-477c-8bde-43d14d6e1ac6","Type":"ContainerStarted","Data":"9a5e4412ddaa255beb5d71668bb156934a911ea5c21fdac23dc7dce34d0b64c7"} Jul 27 23:37:27.336175 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.336136 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvhtj" event={"ID":"66c4f215-ea76-4982-a115-bc31e5efcd77","Type":"ContainerStarted","Data":"67c7e1afbed475ea00373e0b86f30a930cb3341020e0f8e760100c5488297e3a"} Jul 27 23:37:27.338208 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.338183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" event={"ID":"8ef8a9b6-fcb1-4448-9fc6-d728dda2f952","Type":"ContainerStarted","Data":"80e01e0584c31e74fb5497ccabcc2d5259431c434dd19986f65fefd24710591a"} Jul 27 23:37:27.339794 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.339773 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" event={"ID":"57b84b79-a6f8-4c85-ba66-4bd9a964961a","Type":"ContainerStarted","Data":"64ed5a535e1adf9d6c5b1909df520d4bfbc08605cf191406b8699817336c99de"} Jul 27 23:37:27.344541 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.344480 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rm6tp" podStartSLOduration=3.659300594 podStartE2EDuration="22.344469259s" totalImagesPullingTime="18.685168665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:27.344378226 +0000 UTC m=+22.704914904" watchObservedRunningTime="2026-07-27 23:37:27.344469259 +0000 UTC m=+22.705005936" Jul 27 23:37:27.344741 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.344722 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-69.ec2.internal" podStartSLOduration=21.34471787 podStartE2EDuration="21.34471787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:10.30864369 +0000 UTC m=+5.669180373" watchObservedRunningTime="2026-07-27 23:37:27.34471787 +0000 UTC m=+22.705254591" Jul 27 23:37:27.372094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.372032 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xd4ck" podStartSLOduration=3.681557144 podStartE2EDuration="22.372008319s" totalImagesPullingTime="18.690451175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:27.371335776 +0000 UTC m=+22.731872454" watchObservedRunningTime="2026-07-27 23:37:27.372008319 +0000 UTC m=+22.732544998" Jul 27 23:37:27.372378 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.372342 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rhcb4" podStartSLOduration=3.626405902 podStartE2EDuration="22.372328635s" totalImagesPullingTime="18.745922733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:27.357674919 +0000 UTC m=+22.718211597" watchObservedRunningTime="2026-07-27 23:37:27.372328635 +0000 UTC m=+22.732865314" Jul 27 23:37:27.426004 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.425949 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hsc4t" podStartSLOduration=3.671196741 podStartE2EDuration="22.42593392s" totalImagesPullingTime="18.754737179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:27.409480287 +0000 UTC m=+22.770016984" watchObservedRunningTime="2026-07-27 23:37:27.42593392 +0000 UTC m=+22.786470598" Jul 27 23:37:27.426329 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:27.426295 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kvhtj" podStartSLOduration=3.6643064499999998 podStartE2EDuration="22.426286385s" totalImagesPullingTime="18.761979935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:27.425818273 +0000 UTC m=+22.786354951" watchObservedRunningTime="2026-07-27 23:37:27.426286385 +0000 UTC m=+22.786823063" Jul 27 23:37:28.167725 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.167458 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:28.167982 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.167496 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:28.167982 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:28.167841 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:28.167982 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:28.167906 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:28.348472 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.348400 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.348762 2574 generic.go:356] "Generic (PLEG): container finished" podID="5cb08f9e-fb93-44ff-aee9-aa4e38b10d44" containerID="7859914c5b5781085ce187749f355f494a216d9dd79dbe2c9c6fbeabb410665c" exitCode=1 Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.348860 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"91c6e3787f6cb00215620fa8901da2d1e3cd4a863af15b6bf5a5e1bcaabb5a06"} Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.348974 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"25690b14d60e3d0b9d39284a1cd7a2a3b14dee9906a7a058d812b83b96972efc"} Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.349015 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"665ca51e0f2a92cebe47ae98f315d9fb6f8639b6f31b9d2a97503cb01784ad0c"} Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.349060 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"a6d22d1d0620991fc504e974dfb1f684099c71a1ba9eacf15771b31cc2b1eefd"} Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.349094 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerDied","Data":"7859914c5b5781085ce187749f355f494a216d9dd79dbe2c9c6fbeabb410665c"} Jul 27 23:37:28.349434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.349138 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"428e26943642611290cd1bf755a0f6fac2073618bbac3e79ed86f4dda159b760"} Jul 27 23:37:28.625073 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.625047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xd4ck_25a22761-c48a-40c9-a7b7-5206184e6615/dns-node-resolver/0.log" Jul 27 23:37:28.671136 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:28.671106 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 23:37:29.051778 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.051609 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T23:37:28.671125125Z","UUID":"0cc233a6-ac46-4740-85a8-0c5fe0aae31c","Handler":null,"Name":"","Endpoint":""} Jul 27 23:37:29.053259 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.053236 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 23:37:29.053386 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.053281 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 23:37:29.167729 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.167698 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:29.167905 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:29.167824 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:29.354494 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.354420 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" event={"ID":"57b84b79-a6f8-4c85-ba66-4bd9a964961a","Type":"ContainerStarted","Data":"dbb0be8a99dd1c8053ef75918e5f2f1ec0caee8d2b7f7ee2605cd3c37ae5db2b"} Jul 27 23:37:29.356175 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.356145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sj786" event={"ID":"6abb1cb5-2321-4511-84a8-489581835a9a","Type":"ContainerStarted","Data":"057969e206060211cd94daf0d1f1e2358cf9873a422eaadeb450c92951cf336f"} Jul 27 23:37:29.370836 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.370795 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sj786" podStartSLOduration=5.6083113430000004 podStartE2EDuration="24.370780955s" totalImagesPullingTime="18.762469612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:29.370202871 +0000 UTC m=+24.730739550" watchObservedRunningTime="2026-07-27 23:37:29.370780955 +0000 UTC m=+24.731317662" Jul 27 23:37:29.594388 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:29.594362 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rhcb4_6e6dfd4f-793c-477c-8bde-43d14d6e1ac6/node-ca/0.log" Jul 27 23:37:30.167591 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:30.167496 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:30.167750 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:30.167496 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:30.167750 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:30.167629 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:30.167750 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:30.167719 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:30.361241 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:30.361196 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" event={"ID":"57b84b79-a6f8-4c85-ba66-4bd9a964961a","Type":"ContainerStarted","Data":"55de7b75523cba9eed7faba9d0594ef9425e4570477a2557779401a48287ad22"} Jul 27 23:37:30.382828 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:30.382767 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5vd8" podStartSLOduration=4.172332883 podStartE2EDuration="25.382750685s" totalImagesPullingTime="21.210417802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:30.382454614 +0000 UTC m=+25.742991292" watchObservedRunningTime="2026-07-27 23:37:30.382750685 +0000 UTC m=+25.743287363" Jul 27 23:37:31.167432 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:31.167402 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:31.167659 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:31.167538 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:31.220127 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:31.220085 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:31.220781 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:31.220758 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:32.167089 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.167003 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:32.167089 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.167048 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:32.167819 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:32.167121 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:32.167819 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:32.167225 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:32.367351 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.367318 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="3a528204d5e94eadd7b2dd959863337981ef0ae5b4cafa27ef9ebbe065023d5e" exitCode=0 Jul 27 23:37:32.367644 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.367409 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"3a528204d5e94eadd7b2dd959863337981ef0ae5b4cafa27ef9ebbe065023d5e"} Jul 27 23:37:32.370585 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.370568 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:37:32.370952 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:32.370922 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"fac63b7ecd6bb8d1d9190a18f6375b85fbbe158cc4f5c017c69de07df5c616c6"} Jul 27 23:37:33.167299 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:33.167260 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:33.167795 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:33.167376 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:34.166949 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.166713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:34.167110 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.166721 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:34.167110 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:34.166985 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:34.167110 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:34.167059 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:34.378196 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.378161 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="e9928358790bb1025965c019a78a99c8efb40b9002503014a07ac4b12cf5b06d" exitCode=0 Jul 27 23:37:34.378690 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.378224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"e9928358790bb1025965c019a78a99c8efb40b9002503014a07ac4b12cf5b06d"} Jul 27 23:37:34.381894 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.381875 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:37:34.382240 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.382219 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"e042b4f82c6352a8c7c48b36c21812ff94076a5d7d49573cd450f5ff2f8ac5f1"} Jul 27 23:37:34.382565 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.382540 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:34.382644 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.382620 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:34.382792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.382773 2574 scope.go:117] "RemoveContainer" containerID="7859914c5b5781085ce187749f355f494a216d9dd79dbe2c9c6fbeabb410665c" Jul 27 23:37:34.398400 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:34.398378 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:35.167320 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.167292 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:35.167468 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:35.167383 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:35.389435 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.389411 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:37:35.389871 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.389780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" event={"ID":"5cb08f9e-fb93-44ff-aee9-aa4e38b10d44","Type":"ContainerStarted","Data":"9273d8653a55935f7a157a470c7730cd17f89cd5b629a31b7f30d55b411c09d2"} Jul 27 23:37:35.397856 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.397797 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:35.423386 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.423324 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:37:35.425354 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:35.425314 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" podStartSLOduration=11.44642613 podStartE2EDuration="30.42530236s" totalImagesPullingTime="18.97887623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:35.423619744 +0000 UTC m=+30.784156448" watchObservedRunningTime="2026-07-27 23:37:35.42530236 +0000 UTC m=+30.785839038" Jul 27 23:37:36.167447 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.167270 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:36.167628 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.167270 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:36.167628 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:36.167530 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:36.167628 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:36.167597 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:36.267607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.267580 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:36.267774 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.267697 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 27 23:37:36.268161 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.268142 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rm6tp" Jul 27 23:37:36.394952 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.394917 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="2d94d39c3c3e59669bf4ab10d53991148f76d4f9820b26ad438d81a97fe1f16b" exitCode=0 Jul 27 23:37:36.395384 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:36.394997 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"2d94d39c3c3e59669bf4ab10d53991148f76d4f9820b26ad438d81a97fe1f16b"} Jul 27 23:37:37.167301 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:37.167266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:37.167500 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:37.167413 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:38.166609 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:38.166573 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:38.167060 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.166701 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:38.167060 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:38.166765 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:38.167060 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.166876 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:38.751244 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:38.751211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:38.751492 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.751321 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:38.751492 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.751375 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret podName:2c884103-50ba-403b-89eb-66d666438a5c nodeName:}" failed. No retries permitted until 2026-07-27 23:38:10.751360462 +0000 UTC m=+66.111897117 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret") pod "global-pull-secret-syncer-x7j27" (UID: "2c884103-50ba-403b-89eb-66d666438a5c") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:38.851934 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:38.851890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:38.852124 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.852055 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:38.852190 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.852134 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs podName:e704052e-dd94-4bf3-b3a2-1589b7c3ab62 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:10.852113884 +0000 UTC m=+66.212650559 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs") pod "network-metrics-daemon-4w58m" (UID: "e704052e-dd94-4bf3-b3a2-1589b7c3ab62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:38.952651 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:38.952609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:38.952841 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.952783 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:38.952841 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.952814 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:38.952841 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.952826 2574 projected.go:196] Error preparing data for projected volume kube-api-access-74hbg for pod openshift-network-diagnostics/network-check-target-nr6fd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:38.952981 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:38.952883 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg podName:2067dc2d-166c-4f47-aae9-87373917a88f nodeName:}" failed. No retries permitted until 2026-07-27 23:38:10.952864376 +0000 UTC m=+66.313401032 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-74hbg" (UniqueName: "kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg") pod "network-check-target-nr6fd" (UID: "2067dc2d-166c-4f47-aae9-87373917a88f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:39.167067 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:39.167021 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:39.167556 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:39.167182 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:40.167141 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:40.167109 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:40.167624 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:40.167109 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:40.167624 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:40.167237 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:40.167624 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:40.167371 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:40.408532 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:40.408478 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerStarted","Data":"6454ed799dd6fcbe145a04b92e93be1762e3e16183ec42a8af659ed76388fa13"} Jul 27 23:37:41.166896 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:41.166856 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:41.167064 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:41.166990 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:41.416099 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:41.416061 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="6454ed799dd6fcbe145a04b92e93be1762e3e16183ec42a8af659ed76388fa13" exitCode=0 Jul 27 23:37:41.416484 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:41.416126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"6454ed799dd6fcbe145a04b92e93be1762e3e16183ec42a8af659ed76388fa13"} Jul 27 23:37:42.167371 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:42.167338 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:42.167601 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:42.167339 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:42.167601 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:42.167465 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:42.167601 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:42.167506 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:42.421769 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:42.421674 2574 generic.go:356] "Generic (PLEG): container finished" podID="3b755624-4b79-4d54-bfc2-6ac2fd632c5f" containerID="cd9b8e0723cf77a9c183e4f58db27179f7b70e4bd2fa13762a71c87b334de588" exitCode=0 Jul 27 23:37:42.421769 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:42.421733 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerDied","Data":"cd9b8e0723cf77a9c183e4f58db27179f7b70e4bd2fa13762a71c87b334de588"} Jul 27 23:37:43.167653 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:43.167618 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:43.167828 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:43.167730 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:43.426719 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:43.426641 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" event={"ID":"3b755624-4b79-4d54-bfc2-6ac2fd632c5f","Type":"ContainerStarted","Data":"b642188aa7809150b44462370d374942e055aecc08b303a3152984800cfeb97c"} Jul 27 23:37:43.450166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:43.450122 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dzh5j" podStartSLOduration=9.112811373 podStartE2EDuration="38.450107607s" totalImagesPullingTime="29.337296234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:43.448786722 +0000 UTC m=+38.809323399" watchObservedRunningTime="2026-07-27 23:37:43.450107607 +0000 UTC m=+38.810644284" Jul 27 23:37:44.166741 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:44.166710 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:44.166916 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:44.166710 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:44.166916 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:44.166808 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:44.166916 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:44.166874 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:45.167527 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:45.167487 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:45.167901 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:45.167606 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:46.166943 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.166912 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:46.167129 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:46.167038 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:46.167129 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.167114 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:46.167249 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:46.167227 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:46.969269 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.969235 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7j27"] Jul 27 23:37:46.970017 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.969342 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:46.970017 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:46.969452 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:46.972095 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.972069 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4w58m"] Jul 27 23:37:46.972233 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.972165 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:46.972294 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:46.972266 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:46.981362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.981031 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nr6fd"] Jul 27 23:37:46.981362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:46.981117 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:46.981362 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:46.981206 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:48.167577 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:48.167360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:48.168034 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:48.167687 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:49.166852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:49.166820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:49.167019 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:49.166820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:49.167019 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:49.166948 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:49.167019 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:49.166997 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:50.166706 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:50.166676 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:50.167084 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:50.166777 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7j27" podUID="2c884103-50ba-403b-89eb-66d666438a5c" Jul 27 23:37:51.166565 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:51.166531 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:51.166741 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:51.166573 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:51.166741 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:51.166671 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4w58m" podUID="e704052e-dd94-4bf3-b3a2-1589b7c3ab62" Jul 27 23:37:51.167026 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:37:51.166771 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nr6fd" podUID="2067dc2d-166c-4f47-aae9-87373917a88f" Jul 27 23:37:51.975411 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:51.975385 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-69.ec2.internal" event="NodeReady" Jul 27 23:37:51.975590 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:51.975500 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 23:37:52.022259 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.022220 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8mfhf"] Jul 27 23:37:52.043490 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.043462 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t9xjf"] Jul 27 23:37:52.043639 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.043588 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.046333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.046304 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 23:37:52.046333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.046325 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 23:37:52.046503 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.046378 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 23:37:52.046503 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.046378 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 23:37:52.046503 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.046431 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbzb5\"" Jul 27 23:37:52.051277 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.051260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mfhf"] Jul 27 23:37:52.051340 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.051295 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9xjf"] Jul 27 23:37:52.051375 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.051366 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.053839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.053822 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 23:37:52.053982 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.053829 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 23:37:52.053982 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.053878 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jk8w\"" Jul 27 23:37:52.121856 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.121825 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4xlw5"] Jul 27 23:37:52.136954 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.136933 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4xlw5"] Jul 27 23:37:52.137060 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.137020 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.139578 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.139557 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 23:37:52.139688 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.139634 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fz2fq\"" Jul 27 23:37:52.139688 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.139643 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 23:37:52.139790 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.139702 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 23:37:52.150312 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee4bc17-cf15-4918-b9e9-96e58bf78871-crio-socket\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.150446 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1e08ab-c13b-4544-a1b9-b420f86b9670-metrics-tls\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.150542 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1e08ab-c13b-4544-a1b9-b420f86b9670-tmp-dir\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.150608 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee4bc17-cf15-4918-b9e9-96e58bf78871-data-volume\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.150608 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mphq8\" (UniqueName: \"kubernetes.io/projected/ee1e08ab-c13b-4544-a1b9-b420f86b9670-kube-api-access-mphq8\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.150709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150609 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgjgr\" (UniqueName: \"kubernetes.io/projected/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-api-access-hgjgr\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.150709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1e08ab-c13b-4544-a1b9-b420f86b9670-config-volume\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.150804 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150709 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee4bc17-cf15-4918-b9e9-96e58bf78871-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.150804 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.150740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.166975 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.166952 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:37:52.169607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.169591 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 23:37:52.251737 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee4bc17-cf15-4918-b9e9-96e58bf78871-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.251737 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee4bc17-cf15-4918-b9e9-96e58bf78871-crio-socket\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1e08ab-c13b-4544-a1b9-b420f86b9670-metrics-tls\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251780 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fkpn\" (UniqueName: \"kubernetes.io/projected/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-kube-api-access-4fkpn\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1e08ab-c13b-4544-a1b9-b420f86b9670-tmp-dir\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee4bc17-cf15-4918-b9e9-96e58bf78871-data-volume\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251895 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mphq8\" (UniqueName: \"kubernetes.io/projected/ee1e08ab-c13b-4544-a1b9-b420f86b9670-kube-api-access-mphq8\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.251953 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgjgr\" (UniqueName: \"kubernetes.io/projected/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-api-access-hgjgr\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.252333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251973 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee4bc17-cf15-4918-b9e9-96e58bf78871-crio-socket\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.252333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.251984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-cert\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.252333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.252045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1e08ab-c13b-4544-a1b9-b420f86b9670-config-volume\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.252333 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.252220 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1e08ab-c13b-4544-a1b9-b420f86b9670-tmp-dir\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.252537 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.252333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee4bc17-cf15-4918-b9e9-96e58bf78871-data-volume\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.252599 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.252571 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1e08ab-c13b-4544-a1b9-b420f86b9670-config-volume\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.252599 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.252585 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.255764 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.255742 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1e08ab-c13b-4544-a1b9-b420f86b9670-metrics-tls\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.255871 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.255745 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee4bc17-cf15-4918-b9e9-96e58bf78871-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.260035 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.260013 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgjgr\" (UniqueName: \"kubernetes.io/projected/eee4bc17-cf15-4918-b9e9-96e58bf78871-kube-api-access-hgjgr\") pod \"insights-runtime-extractor-8mfhf\" (UID: \"eee4bc17-cf15-4918-b9e9-96e58bf78871\") " pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.260148 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.260038 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mphq8\" (UniqueName: \"kubernetes.io/projected/ee1e08ab-c13b-4544-a1b9-b420f86b9670-kube-api-access-mphq8\") pod \"dns-default-t9xjf\" (UID: \"ee1e08ab-c13b-4544-a1b9-b420f86b9670\") " pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.352578 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.352539 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mfhf" Jul 27 23:37:52.352749 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.352674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fkpn\" (UniqueName: \"kubernetes.io/projected/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-kube-api-access-4fkpn\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.352749 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.352729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-cert\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.355046 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.355018 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-cert\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.359002 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.358980 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9xjf" Jul 27 23:37:52.360846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.360799 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fkpn\" (UniqueName: \"kubernetes.io/projected/6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c-kube-api-access-4fkpn\") pod \"ingress-canary-4xlw5\" (UID: \"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c\") " pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.445016 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.444987 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4xlw5" Jul 27 23:37:52.566371 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.566270 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mfhf"] Jul 27 23:37:52.569075 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.569055 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9xjf"] Jul 27 23:37:52.571332 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:52.571299 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee4bc17_cf15_4918_b9e9_96e58bf78871.slice/crio-7a60892abf24601848af54e54bf94755cf763b80ccd05ac7ca48b279e52610b9 WatchSource:0}: Error finding container 7a60892abf24601848af54e54bf94755cf763b80ccd05ac7ca48b279e52610b9: Status 404 returned error can't find the container with id 7a60892abf24601848af54e54bf94755cf763b80ccd05ac7ca48b279e52610b9 Jul 27 23:37:52.571896 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:52.571872 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1e08ab_c13b_4544_a1b9_b420f86b9670.slice/crio-201b277a03176dd71c64a0e6fc51343ef159071832b1a761c0ff1bd8a1ca7625 WatchSource:0}: Error finding container 201b277a03176dd71c64a0e6fc51343ef159071832b1a761c0ff1bd8a1ca7625: Status 404 returned error can't find the container with id 201b277a03176dd71c64a0e6fc51343ef159071832b1a761c0ff1bd8a1ca7625 Jul 27 23:37:52.605852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:52.605817 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4xlw5"] Jul 27 23:37:52.610389 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:52.610360 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea3a782_5d0f_4ec5_aeb6_8e54c1fe083c.slice/crio-75f63581d92777a4e17c4d937f46b55e0f3a692b35a73dd4dbc34263812e05b9 WatchSource:0}: Error finding container 75f63581d92777a4e17c4d937f46b55e0f3a692b35a73dd4dbc34263812e05b9: Status 404 returned error can't find the container with id 75f63581d92777a4e17c4d937f46b55e0f3a692b35a73dd4dbc34263812e05b9 Jul 27 23:37:53.166915 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.166881 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:37:53.167096 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.166889 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:37:53.170987 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.170962 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 23:37:53.171128 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.170966 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t92bh\"" Jul 27 23:37:53.171410 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.171371 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 23:37:53.171410 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.171372 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 23:37:53.171601 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.171386 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p7f9\"" Jul 27 23:37:53.451027 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.450941 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4xlw5" event={"ID":"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c","Type":"ContainerStarted","Data":"75f63581d92777a4e17c4d937f46b55e0f3a692b35a73dd4dbc34263812e05b9"} Jul 27 23:37:53.452459 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.452429 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9xjf" event={"ID":"ee1e08ab-c13b-4544-a1b9-b420f86b9670","Type":"ContainerStarted","Data":"201b277a03176dd71c64a0e6fc51343ef159071832b1a761c0ff1bd8a1ca7625"} Jul 27 23:37:53.480762 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.480593 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mfhf" event={"ID":"eee4bc17-cf15-4918-b9e9-96e58bf78871","Type":"ContainerStarted","Data":"7a60892abf24601848af54e54bf94755cf763b80ccd05ac7ca48b279e52610b9"} Jul 27 23:37:53.544446 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.543404 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:37:53.566142 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.566084 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:37:53.566308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.566194 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.569190 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.569165 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zfr5x\"" Jul 27 23:37:53.569331 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.569260 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 27 23:37:53.569452 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.569435 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 27 23:37:53.569506 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.569462 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 27 23:37:53.571915 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.571893 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 27 23:37:53.572167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.572149 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 27 23:37:53.572360 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.572345 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 27 23:37:53.572589 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.572572 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669011 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlpk2\" (UniqueName: \"kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.669433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.669296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770132 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.770258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlpk2\" (UniqueName: \"kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.771582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.771346 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.772424 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.772382 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.772561 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.772541 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.776343 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.776301 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.778579 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.778536 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlpk2\" (UniqueName: \"kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.779508 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.779484 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert\") pod \"console-6dc8c94bd4-tpzrb\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:53.883655 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:53.883373 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:37:54.736074 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:54.735998 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:37:55.693332 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:55.693300 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b546693_11ae_4e87_87d6_556009964b26.slice/crio-65be2c6b5bfbf7a08e3dd7d450b87d9edfbab4d1728f5b3df309c215524b388e WatchSource:0}: Error finding container 65be2c6b5bfbf7a08e3dd7d450b87d9edfbab4d1728f5b3df309c215524b388e: Status 404 returned error can't find the container with id 65be2c6b5bfbf7a08e3dd7d450b87d9edfbab4d1728f5b3df309c215524b388e Jul 27 23:37:56.491212 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:56.491182 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8c94bd4-tpzrb" event={"ID":"8b546693-11ae-4e87-87d6-556009964b26","Type":"ContainerStarted","Data":"65be2c6b5bfbf7a08e3dd7d450b87d9edfbab4d1728f5b3df309c215524b388e"} Jul 27 23:37:57.749820 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:57.748036 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq"] Jul 27 23:37:58.020004 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.019911 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq"] Jul 27 23:37:58.020252 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.020028 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:37:58.023711 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.023691 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 27 23:37:58.023839 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.023720 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-q8972\"" Jul 27 23:37:58.108651 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.108614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0890dc4-52fc-4e40-a36f-4ba2b4651f56-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-w6tmq\" (UID: \"e0890dc4-52fc-4e40-a36f-4ba2b4651f56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:37:58.209101 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.209067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0890dc4-52fc-4e40-a36f-4ba2b4651f56-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-w6tmq\" (UID: \"e0890dc4-52fc-4e40-a36f-4ba2b4651f56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:37:58.211800 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.211780 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0890dc4-52fc-4e40-a36f-4ba2b4651f56-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-w6tmq\" (UID: \"e0890dc4-52fc-4e40-a36f-4ba2b4651f56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:37:58.329305 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:58.329227 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:37:59.506484 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:59.506015 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4xlw5" event={"ID":"6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c","Type":"ContainerStarted","Data":"49238f896c6905a24045504ea2e6f84a49c306b7d4cc9a4f88254cea0f5fe036"} Jul 27 23:37:59.508759 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:59.508558 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq"] Jul 27 23:37:59.510304 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:59.510223 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9xjf" event={"ID":"ee1e08ab-c13b-4544-a1b9-b420f86b9670","Type":"ContainerStarted","Data":"37014c7b43dfee374093aed650198e36cf1926a19bd3e0611562f825630d48dc"} Jul 27 23:37:59.520199 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:37:59.520146 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4xlw5" podStartSLOduration=0.766071169 podStartE2EDuration="7.520128163s" totalImagesPullingTime="6.754056994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:59.519161433 +0000 UTC m=+54.879698113" watchObservedRunningTime="2026-07-27 23:37:59.520128163 +0000 UTC m=+54.880664823" Jul 27 23:37:59.547295 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:37:59.547032 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0890dc4_52fc_4e40_a36f_4ba2b4651f56.slice/crio-7bc590c3022ed811790cb29c67c824eace0cf39ee7bddfb784881ddcbcf6f5f6 WatchSource:0}: Error finding container 7bc590c3022ed811790cb29c67c824eace0cf39ee7bddfb784881ddcbcf6f5f6: Status 404 returned error can't find the container with id 7bc590c3022ed811790cb29c67c824eace0cf39ee7bddfb784881ddcbcf6f5f6 Jul 27 23:38:00.515475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.515434 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mfhf" event={"ID":"eee4bc17-cf15-4918-b9e9-96e58bf78871","Type":"ContainerStarted","Data":"10f3685d254321dde30d416a115d2edea84888116ec02825ff1bb1da98a48d34"} Jul 27 23:38:00.516884 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.516857 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" event={"ID":"e0890dc4-52fc-4e40-a36f-4ba2b4651f56","Type":"ContainerStarted","Data":"7bc590c3022ed811790cb29c67c824eace0cf39ee7bddfb784881ddcbcf6f5f6"} Jul 27 23:38:00.518972 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.518943 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9xjf" event={"ID":"ee1e08ab-c13b-4544-a1b9-b420f86b9670","Type":"ContainerStarted","Data":"ae96308b937b1cc539918dff98822565bacc987bf58180f31e25b030b2e8cda5"} Jul 27 23:38:00.537868 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.537810 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t9xjf" podStartSLOduration=1.747121268 podStartE2EDuration="8.537788959s" totalImagesPullingTime="6.790667691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:00.537174153 +0000 UTC m=+55.897710855" watchObservedRunningTime="2026-07-27 23:38:00.537788959 +0000 UTC m=+55.898325638" Jul 27 23:38:00.541716 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.541692 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:00.570392 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.570358 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:00.570564 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.570444 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.581354 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.578285 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 27 23:38:00.626095 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626059 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626281 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626154 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626281 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626181 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2n7\" (UniqueName: \"kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626281 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626281 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626659 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.626659 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.626337 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727327 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727327 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727364 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727445 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2n7\" (UniqueName: \"kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727526 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.727612 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.727554 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.728652 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.728602 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.728755 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.728692 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.728755 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.728701 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.728755 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.728727 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.732280 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.732254 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.732411 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.732334 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.736980 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.736957 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2n7\" (UniqueName: \"kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7\") pod \"console-db94477dd-phmlm\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:00.884175 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:00.884135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:01.171891 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:01.171825 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:01.174448 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:01.174420 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb7cd3c_6c0d_4047_9ccb_975cbd682a0f.slice/crio-e62a4c3b3ad38cf9ac7c582210ecc744f834552f3efc1aab4abd9cee0ef429d3 WatchSource:0}: Error finding container e62a4c3b3ad38cf9ac7c582210ecc744f834552f3efc1aab4abd9cee0ef429d3: Status 404 returned error can't find the container with id e62a4c3b3ad38cf9ac7c582210ecc744f834552f3efc1aab4abd9cee0ef429d3 Jul 27 23:38:01.528902 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:01.528254 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db94477dd-phmlm" event={"ID":"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f","Type":"ContainerStarted","Data":"e62a4c3b3ad38cf9ac7c582210ecc744f834552f3efc1aab4abd9cee0ef429d3"} Jul 27 23:38:01.531348 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:01.531317 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8c94bd4-tpzrb" event={"ID":"8b546693-11ae-4e87-87d6-556009964b26","Type":"ContainerStarted","Data":"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f"} Jul 27 23:38:01.531469 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:01.531395 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t9xjf" Jul 27 23:38:02.536789 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:02.536752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db94477dd-phmlm" event={"ID":"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f","Type":"ContainerStarted","Data":"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811"} Jul 27 23:38:02.538231 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:02.538204 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" event={"ID":"e0890dc4-52fc-4e40-a36f-4ba2b4651f56","Type":"ContainerStarted","Data":"2fbf2ab827d27abe78a705849f1f18119ea932fb06dcb712550be66b54eff257"} Jul 27 23:38:02.553819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:02.553772 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-db94477dd-phmlm" podStartSLOduration=1.62889084 podStartE2EDuration="2.553756187s" totalImagesPullingTime="924.865347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:02.552125931 +0000 UTC m=+57.912662609" watchObservedRunningTime="2026-07-27 23:38:02.553756187 +0000 UTC m=+57.914292844" Jul 27 23:38:02.566479 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:02.566435 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" podStartSLOduration=2.836486302 podStartE2EDuration="5.566419527s" totalImagesPullingTime="2.729933225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:02.565840235 +0000 UTC m=+57.926376912" watchObservedRunningTime="2026-07-27 23:38:02.566419527 +0000 UTC m=+57.926956182" Jul 27 23:38:02.587590 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:02.587431 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dc8c94bd4-tpzrb" podStartSLOduration=3.873374358 podStartE2EDuration="9.58741448s" totalImagesPullingTime="5.714040122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:02.587367378 +0000 UTC m=+57.947904055" watchObservedRunningTime="2026-07-27 23:38:02.58741448 +0000 UTC m=+57.947951159" Jul 27 23:38:03.544229 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.544119 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:38:03.550216 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.550186 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-w6tmq" Jul 27 23:38:03.829282 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.829206 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ksvrv"] Jul 27 23:38:03.831167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.831149 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.834926 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.834946 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.834968 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hmfdv\"" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.834929 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.834990 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 27 23:38:03.835023 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.835009 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 23:38:03.841212 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.841187 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ksvrv"] Jul 27 23:38:03.851718 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.851683 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.851860 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.851732 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.851860 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.851778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6sww\" (UniqueName: \"kubernetes.io/projected/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-kube-api-access-s6sww\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.851860 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.851806 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.884402 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.884360 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:38:03.884600 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.884430 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:38:03.885868 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.885843 2574 patch_prober.go:28] interesting pod/console-6dc8c94bd4-tpzrb container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 27 23:38:03.885954 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.885891 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6dc8c94bd4-tpzrb" podUID="8b546693-11ae-4e87-87d6-556009964b26" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 27 23:38:03.952897 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.952855 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.953087 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.952917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.953087 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.953007 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6sww\" (UniqueName: \"kubernetes.io/projected/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-kube-api-access-s6sww\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.953087 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:03.953017 2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 27 23:38:03.953087 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.953044 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.953276 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:03.953092 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls podName:ed1681b0-306c-4ff1-a304-54a1f2aa3cd7 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:04.4530691 +0000 UTC m=+59.813605808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-ksvrv" (UID: "ed1681b0-306c-4ff1-a304-54a1f2aa3cd7") : secret "prometheus-operator-tls" not found Jul 27 23:38:03.953802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.953776 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.956699 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.956678 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:03.962025 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:03.961996 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6sww\" (UniqueName: \"kubernetes.io/projected/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-kube-api-access-s6sww\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:04.458234 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:04.458196 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:04.460770 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:04.460743 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed1681b0-306c-4ff1-a304-54a1f2aa3cd7-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ksvrv\" (UID: \"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:04.741540 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:04.741439 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" Jul 27 23:38:04.861475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:04.861422 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ksvrv"] Jul 27 23:38:04.865093 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:04.865065 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1681b0_306c_4ff1_a304_54a1f2aa3cd7.slice/crio-d599de0dcf41d5747c1b09e15ed8036fd466479cca377a9f2079dcb77b895188 WatchSource:0}: Error finding container d599de0dcf41d5747c1b09e15ed8036fd466479cca377a9f2079dcb77b895188: Status 404 returned error can't find the container with id d599de0dcf41d5747c1b09e15ed8036fd466479cca377a9f2079dcb77b895188 Jul 27 23:38:05.551430 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:05.551388 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mfhf" event={"ID":"eee4bc17-cf15-4918-b9e9-96e58bf78871","Type":"ContainerStarted","Data":"41fbd74601ceb58614b49f95b9d4f8082c40dcb5451a2f5cd490873e6259d514"} Jul 27 23:38:05.552947 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:05.552916 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" event={"ID":"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7","Type":"ContainerStarted","Data":"d599de0dcf41d5747c1b09e15ed8036fd466479cca377a9f2079dcb77b895188"} Jul 27 23:38:06.559021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:06.558936 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" event={"ID":"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7","Type":"ContainerStarted","Data":"b8e71003b26e74839699a61637810b7431c17534cdd1fd4486e19bd51c29f28d"} Jul 27 23:38:06.559021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:06.558998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" event={"ID":"ed1681b0-306c-4ff1-a304-54a1f2aa3cd7","Type":"ContainerStarted","Data":"247603a9f4fef32a1fc0c0e9e24e30305ade9eec91f4fe0c5eb25f92e7daa7ce"} Jul 27 23:38:06.575641 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:06.575586 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-ksvrv" podStartSLOduration=2.199778953 podStartE2EDuration="3.575565216s" totalImagesPullingTime="1.375786263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:06.574657725 +0000 UTC m=+61.935194404" watchObservedRunningTime="2026-07-27 23:38:06.575565216 +0000 UTC m=+61.936101896" Jul 27 23:38:07.409574 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:07.409543 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksj56" Jul 27 23:38:08.183649 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.183616 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h"] Jul 27 23:38:08.185785 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.185765 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.188213 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.188193 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 27 23:38:08.188336 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.188227 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9rkqq\"" Jul 27 23:38:08.189771 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.189753 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 27 23:38:08.196879 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.196859 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h"] Jul 27 23:38:08.199937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.199915 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-94hps"] Jul 27 23:38:08.202400 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.202382 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nkcxr"] Jul 27 23:38:08.202495 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.202479 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.204715 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.204700 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 27 23:38:08.204793 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.204736 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.205011 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.204973 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 27 23:38:08.205114 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.205085 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-k494z\"" Jul 27 23:38:08.205185 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.205159 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 27 23:38:08.206674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.206659 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 23:38:08.207312 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.207284 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7v7md\"" Jul 27 23:38:08.207468 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.207450 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 23:38:08.207618 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.207601 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 23:38:08.213230 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.213212 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-94hps"] Jul 27 23:38:08.295197 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295197 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295196 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccxwl\" (UniqueName: \"kubernetes.io/projected/f84fff22-f1c7-471d-be60-0baf1ca651a6-kube-api-access-ccxwl\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jdmc\" (UniqueName: \"kubernetes.io/projected/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-api-access-9jdmc\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295291 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-metrics-client-ca\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295365 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9293afef-0bec-450f-a54c-37ee3bbb889c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.295426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg28w\" (UniqueName: \"kubernetes.io/projected/9293afef-0bec-450f-a54c-37ee3bbb889c-kube-api-access-jg28w\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295448 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57ef9b32-ce47-481b-b23d-550c82bcc50c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-root\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295542 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-wtmp\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295624 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-textfile\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295659 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.295805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-sys\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295714 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.295805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295742 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.295805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.295774 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.396705 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9293afef-0bec-450f-a54c-37ee3bbb889c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.396705 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396708 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg28w\" (UniqueName: \"kubernetes.io/projected/9293afef-0bec-450f-a54c-37ee3bbb889c-kube-api-access-jg28w\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57ef9b32-ce47-481b-b23d-550c82bcc50c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-root\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396855 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-wtmp\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-textfile\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396925 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-root\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.396964 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.396943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-sys\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397110 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397129 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-wtmp\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397141 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397215 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccxwl\" (UniqueName: \"kubernetes.io/projected/f84fff22-f1c7-471d-be60-0baf1ca651a6-kube-api-access-ccxwl\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397225 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57ef9b32-ce47-481b-b23d-550c82bcc50c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jdmc\" (UniqueName: \"kubernetes.io/projected/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-api-access-9jdmc\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397279 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.397412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-metrics-client-ca\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9293afef-0bec-450f-a54c-37ee3bbb889c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-textfile\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:08.397603 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:08.397658 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls podName:f84fff22-f1c7-471d-be60-0baf1ca651a6 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:08.897640822 +0000 UTC m=+64.258177480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls") pod "node-exporter-nkcxr" (UID: "f84fff22-f1c7-471d-be60-0baf1ca651a6") : secret "node-exporter-tls" not found Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.397818 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.398272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.398082 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f84fff22-f1c7-471d-be60-0baf1ca651a6-sys\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.398607 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:08.398353 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 27 23:38:08.398607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.398372 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ef9b32-ce47-481b-b23d-550c82bcc50c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.398607 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:08.398410 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls podName:57ef9b32-ce47-481b-b23d-550c82bcc50c nodeName:}" failed. No retries permitted until 2026-07-27 23:38:08.898394994 +0000 UTC m=+64.258931664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-94hps" (UID: "57ef9b32-ce47-481b-b23d-550c82bcc50c") : secret "kube-state-metrics-tls" not found Jul 27 23:38:08.398607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.398439 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.398607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.398594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f84fff22-f1c7-471d-be60-0baf1ca651a6-metrics-client-ca\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.399824 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.399800 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.399920 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.399808 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9293afef-0bec-450f-a54c-37ee3bbb889c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.400087 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.400064 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.400193 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.400178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.408657 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.408632 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg28w\" (UniqueName: \"kubernetes.io/projected/9293afef-0bec-450f-a54c-37ee3bbb889c-kube-api-access-jg28w\") pod \"openshift-state-metrics-b8f49f5c6-dl57h\" (UID: \"9293afef-0bec-450f-a54c-37ee3bbb889c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.411935 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.411909 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccxwl\" (UniqueName: \"kubernetes.io/projected/f84fff22-f1c7-471d-be60-0baf1ca651a6-kube-api-access-ccxwl\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.412202 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.412186 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jdmc\" (UniqueName: \"kubernetes.io/projected/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-api-access-9jdmc\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.494409 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.494367 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" Jul 27 23:38:08.568196 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.568156 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mfhf" event={"ID":"eee4bc17-cf15-4918-b9e9-96e58bf78871","Type":"ContainerStarted","Data":"de44de5d38caa581927b08aa9bbc56702bbb9d4993e307c656d82efbffcac718"} Jul 27 23:38:08.586659 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.586608 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8mfhf" podStartSLOduration=1.241464203 podStartE2EDuration="16.586591764s" totalImagesPullingTime="15.345127561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:08.585883996 +0000 UTC m=+63.946420720" watchObservedRunningTime="2026-07-27 23:38:08.586591764 +0000 UTC m=+63.947128442" Jul 27 23:38:08.617401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.617369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h"] Jul 27 23:38:08.620552 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:08.620524 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9293afef_0bec_450f_a54c_37ee3bbb889c.slice/crio-0fbc42198178295deb3b94d9032617a1a8d220f53ed86dc94f0c0c7d90f96bca WatchSource:0}: Error finding container 0fbc42198178295deb3b94d9032617a1a8d220f53ed86dc94f0c0c7d90f96bca: Status 404 returned error can't find the container with id 0fbc42198178295deb3b94d9032617a1a8d220f53ed86dc94f0c0c7d90f96bca Jul 27 23:38:08.902035 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.901998 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.902210 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.902059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:08.904405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.904340 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f84fff22-f1c7-471d-be60-0baf1ca651a6-node-exporter-tls\") pod \"node-exporter-nkcxr\" (UID: \"f84fff22-f1c7-471d-be60-0baf1ca651a6\") " pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:08.904405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:08.904393 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57ef9b32-ce47-481b-b23d-550c82bcc50c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-94hps\" (UID: \"57ef9b32-ce47-481b-b23d-550c82bcc50c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:09.112420 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.112385 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" Jul 27 23:38:09.117702 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.117685 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nkcxr" Jul 27 23:38:09.127853 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:09.127812 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84fff22_f1c7_471d_be60_0baf1ca651a6.slice/crio-11b3eb7fecc4261881bb3b20016ef389c8090d0e62108179f00fc28dd3fde205 WatchSource:0}: Error finding container 11b3eb7fecc4261881bb3b20016ef389c8090d0e62108179f00fc28dd3fde205: Status 404 returned error can't find the container with id 11b3eb7fecc4261881bb3b20016ef389c8090d0e62108179f00fc28dd3fde205 Jul 27 23:38:09.244776 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.244742 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 23:38:09.251202 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.251154 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-94hps"] Jul 27 23:38:09.251282 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.251264 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.253464 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.253438 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 27 23:38:09.253723 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.253696 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 27 23:38:09.253993 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.253964 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 27 23:38:09.254083 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.253999 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 27 23:38:09.254139 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.253999 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 27 23:38:09.254139 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.254119 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 27 23:38:09.254400 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.254375 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 27 23:38:09.254533 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.254436 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t7f82\"" Jul 27 23:38:09.254533 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.254472 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 27 23:38:09.254687 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:09.254642 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ef9b32_ce47_481b_b23d_550c82bcc50c.slice/crio-d4c701c7545b96dde746ac991f9bf42cf53f2f42b6463caca5ba9d2faf052217 WatchSource:0}: Error finding container d4c701c7545b96dde746ac991f9bf42cf53f2f42b6463caca5ba9d2faf052217: Status 404 returned error can't find the container with id d4c701c7545b96dde746ac991f9bf42cf53f2f42b6463caca5ba9d2faf052217 Jul 27 23:38:09.254687 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.254677 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 27 23:38:09.261965 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.261880 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 23:38:09.305162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-config-volume\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305185 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8vk\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-kube-api-access-zd8vk\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305209 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-config-out\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305359 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305415 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305401 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-web-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305459 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.305732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.305638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.406832 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.406799 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.406832 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.406836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407068 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.406862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407068 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:09.406976 2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 27 23:38:09.407068 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:09.407039 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls podName:beb23daf-dbc2-49b5-9767-675995599626 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:09.907019502 +0000 UTC m=+65.267556182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "beb23daf-dbc2-49b5-9767-675995599626") : secret "alertmanager-main-tls" not found Jul 27 23:38:09.407223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407153 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407276 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407224 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-config-volume\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407276 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407379 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8vk\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-kube-api-access-zd8vk\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407379 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-config-out\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407379 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407546 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407546 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407546 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407460 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-web-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407546 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407273 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407763 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.407551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.407763 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:09.407682 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle podName:beb23daf-dbc2-49b5-9767-675995599626 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:09.907665616 +0000 UTC m=+65.268202315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "beb23daf-dbc2-49b5-9767-675995599626") : configmap references non-existent config key: ca-bundle.crt Jul 27 23:38:09.410896 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.410406 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.410896 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.410684 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.410896 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.410829 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.411843 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.411813 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.411948 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.411925 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.411948 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.411936 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-config-volume\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.412158 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.412137 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.412158 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.412152 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-web-config\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.412834 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.412216 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beb23daf-dbc2-49b5-9767-675995599626-config-out\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.418625 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.418479 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8vk\" (UniqueName: \"kubernetes.io/projected/beb23daf-dbc2-49b5-9767-675995599626-kube-api-access-zd8vk\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.574182 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.574097 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nkcxr" event={"ID":"f84fff22-f1c7-471d-be60-0baf1ca651a6","Type":"ContainerStarted","Data":"11b3eb7fecc4261881bb3b20016ef389c8090d0e62108179f00fc28dd3fde205"} Jul 27 23:38:09.575780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.575748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" event={"ID":"57ef9b32-ce47-481b-b23d-550c82bcc50c","Type":"ContainerStarted","Data":"d4c701c7545b96dde746ac991f9bf42cf53f2f42b6463caca5ba9d2faf052217"} Jul 27 23:38:09.578078 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.578045 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" event={"ID":"9293afef-0bec-450f-a54c-37ee3bbb889c","Type":"ContainerStarted","Data":"8b97bf8dcc83e25a675c978fad68ff41910c8d485af69ae1f5b6d78269bf43ff"} Jul 27 23:38:09.578191 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.578118 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" event={"ID":"9293afef-0bec-450f-a54c-37ee3bbb889c","Type":"ContainerStarted","Data":"b367b4efc5614cfeee1ad38ff52ae92757d83ce9c70af979fefca5922609fa87"} Jul 27 23:38:09.578191 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.578148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" event={"ID":"9293afef-0bec-450f-a54c-37ee3bbb889c","Type":"ContainerStarted","Data":"0fbc42198178295deb3b94d9032617a1a8d220f53ed86dc94f0c0c7d90f96bca"} Jul 27 23:38:09.911469 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.911425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.911677 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.911535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.912386 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.912358 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb23daf-dbc2-49b5-9767-675995599626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:09.914355 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:09.914328 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beb23daf-dbc2-49b5-9767-675995599626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beb23daf-dbc2-49b5-9767-675995599626\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:10.168364 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.168334 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 23:38:10.319538 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.319432 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 23:38:10.323140 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:10.323113 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb23daf_dbc2_49b5_9767_675995599626.slice/crio-ba4d6d997186a73e114d939f0e93f8497be07a3b445bd41960f66e8bc4caa9a2 WatchSource:0}: Error finding container ba4d6d997186a73e114d939f0e93f8497be07a3b445bd41960f66e8bc4caa9a2: Status 404 returned error can't find the container with id ba4d6d997186a73e114d939f0e93f8497be07a3b445bd41960f66e8bc4caa9a2 Jul 27 23:38:10.583814 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.583723 2574 generic.go:356] "Generic (PLEG): container finished" podID="f84fff22-f1c7-471d-be60-0baf1ca651a6" containerID="17686e9a548e7c54c7ca58c00854220ec83710a111dfccb707d3044229a42522" exitCode=0 Jul 27 23:38:10.583982 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.583816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nkcxr" event={"ID":"f84fff22-f1c7-471d-be60-0baf1ca651a6","Type":"ContainerDied","Data":"17686e9a548e7c54c7ca58c00854220ec83710a111dfccb707d3044229a42522"} Jul 27 23:38:10.585325 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.585292 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"ba4d6d997186a73e114d939f0e93f8497be07a3b445bd41960f66e8bc4caa9a2"} Jul 27 23:38:10.587796 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.587768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" event={"ID":"9293afef-0bec-450f-a54c-37ee3bbb889c","Type":"ContainerStarted","Data":"e87de66121f6d53769624de74365f3efd0611c824fb8a6dd8dd01123826fe8fe"} Jul 27 23:38:10.616588 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.616529 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dl57h" podStartSLOduration=1.557757528 podStartE2EDuration="2.616497704s" totalImagesPullingTime="1.058740176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:10.615177099 +0000 UTC m=+65.975713777" watchObservedRunningTime="2026-07-27 23:38:10.616497704 +0000 UTC m=+65.977034657" Jul 27 23:38:10.833352 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.833316 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:38:10.839450 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.839383 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 23:38:10.846492 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.846437 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c884103-50ba-403b-89eb-66d666438a5c-original-pull-secret\") pod \"global-pull-secret-syncer-x7j27\" (UID: \"2c884103-50ba-403b-89eb-66d666438a5c\") " pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:38:10.884619 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.884595 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:10.884708 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.884701 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:10.886727 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.886150 2574 patch_prober.go:28] interesting pod/console-db94477dd-phmlm container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 27 23:38:10.886727 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.886202 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-db94477dd-phmlm" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 27 23:38:10.934477 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.934440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:38:10.936835 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.936813 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 23:38:10.946709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:10.946684 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e704052e-dd94-4bf3-b3a2-1589b7c3ab62-metrics-certs\") pod \"network-metrics-daemon-4w58m\" (UID: \"e704052e-dd94-4bf3-b3a2-1589b7c3ab62\") " pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:38:11.035929 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.035891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:38:11.038401 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.038374 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 23:38:11.048446 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.048408 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 23:38:11.059285 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.059258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74hbg\" (UniqueName: \"kubernetes.io/projected/2067dc2d-166c-4f47-aae9-87373917a88f-kube-api-access-74hbg\") pod \"network-check-target-nr6fd\" (UID: \"2067dc2d-166c-4f47-aae9-87373917a88f\") " pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:38:11.075095 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.075065 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7j27" Jul 27 23:38:11.182493 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.182460 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p7f9\"" Jul 27 23:38:11.187731 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.187708 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t92bh\"" Jul 27 23:38:11.191105 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.191064 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:38:11.196301 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.196075 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4w58m" Jul 27 23:38:11.236337 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.235982 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7j27"] Jul 27 23:38:11.242184 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:11.242151 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c884103_50ba_403b_89eb_66d666438a5c.slice/crio-9c00dec0c2590a22e6237eb35effa54e6f299851a4c85350016c021d00fb4573 WatchSource:0}: Error finding container 9c00dec0c2590a22e6237eb35effa54e6f299851a4c85350016c021d00fb4573: Status 404 returned error can't find the container with id 9c00dec0c2590a22e6237eb35effa54e6f299851a4c85350016c021d00fb4573 Jul 27 23:38:11.266397 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.266365 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64d66f84d-2s9vf"] Jul 27 23:38:11.271677 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.269917 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.272972 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.273268 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.273485 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gt6xn\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.273725 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.274207 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2iurlkeqij1jl\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.274251 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 27 23:38:11.275166 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.274207 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 27 23:38:11.281423 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.281395 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64d66f84d-2s9vf"] Jul 27 23:38:11.338124 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338124 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338101 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5195fb0-43a0-4373-a1f9-68a8a504d511-metrics-client-ca\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338213 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338242 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfbwq\" (UniqueName: \"kubernetes.io/projected/b5195fb0-43a0-4373-a1f9-68a8a504d511-kube-api-access-vfbwq\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.338622 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.338357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-grpc-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.364565 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.364492 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4w58m"] Jul 27 23:38:11.384317 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.384275 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nr6fd"] Jul 27 23:38:11.439842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439768 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.439842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439802 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5195fb0-43a0-4373-a1f9-68a8a504d511-metrics-client-ca\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.439842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439835 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.440094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439878 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.440094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.440094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439920 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.440094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.439952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfbwq\" (UniqueName: \"kubernetes.io/projected/b5195fb0-43a0-4373-a1f9-68a8a504d511-kube-api-access-vfbwq\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.440094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.440035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-grpc-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.441164 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.441128 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5195fb0-43a0-4373-a1f9-68a8a504d511-metrics-client-ca\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.443122 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.443098 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.443219 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.443180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.443219 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.443198 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-grpc-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.443371 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.443350 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-tls\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.443493 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.443380 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.444043 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.444020 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5195fb0-43a0-4373-a1f9-68a8a504d511-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.447856 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.447836 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfbwq\" (UniqueName: \"kubernetes.io/projected/b5195fb0-43a0-4373-a1f9-68a8a504d511-kube-api-access-vfbwq\") pod \"thanos-querier-64d66f84d-2s9vf\" (UID: \"b5195fb0-43a0-4373-a1f9-68a8a504d511\") " pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.540582 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.540546 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t9xjf" Jul 27 23:38:11.577485 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:11.577443 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode704052e_dd94_4bf3_b3a2_1589b7c3ab62.slice/crio-5aef37e996c99ac778e2c68f7c6baa657f022db20a95e47351a78b6cea4cf019 WatchSource:0}: Error finding container 5aef37e996c99ac778e2c68f7c6baa657f022db20a95e47351a78b6cea4cf019: Status 404 returned error can't find the container with id 5aef37e996c99ac778e2c68f7c6baa657f022db20a95e47351a78b6cea4cf019 Jul 27 23:38:11.578822 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:11.578792 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2067dc2d_166c_4f47_aae9_87373917a88f.slice/crio-2c7fcd9335e0457b75c2d5215fae09eb2ecabcc36a58a4101fb86ef33152c1ed WatchSource:0}: Error finding container 2c7fcd9335e0457b75c2d5215fae09eb2ecabcc36a58a4101fb86ef33152c1ed: Status 404 returned error can't find the container with id 2c7fcd9335e0457b75c2d5215fae09eb2ecabcc36a58a4101fb86ef33152c1ed Jul 27 23:38:11.591085 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.591047 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:11.594388 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.594342 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nkcxr" event={"ID":"f84fff22-f1c7-471d-be60-0baf1ca651a6","Type":"ContainerStarted","Data":"c5073d7dde6070011676cf34d53bfd7940c419300244abc747acc3917f1e0482"} Jul 27 23:38:11.594497 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.594406 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nkcxr" event={"ID":"f84fff22-f1c7-471d-be60-0baf1ca651a6","Type":"ContainerStarted","Data":"70c03b762d5e45d9724cb58eae5d892fdb124cda7a236eda7aab1215cf923201"} Jul 27 23:38:11.595782 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.595757 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nr6fd" event={"ID":"2067dc2d-166c-4f47-aae9-87373917a88f","Type":"ContainerStarted","Data":"2c7fcd9335e0457b75c2d5215fae09eb2ecabcc36a58a4101fb86ef33152c1ed"} Jul 27 23:38:11.598255 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.598227 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" event={"ID":"57ef9b32-ce47-481b-b23d-550c82bcc50c","Type":"ContainerStarted","Data":"c893403118c8e442748b77e3ee545a642b8c07f254ec93618ff394edc6ac405e"} Jul 27 23:38:11.598377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.598290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" event={"ID":"57ef9b32-ce47-481b-b23d-550c82bcc50c","Type":"ContainerStarted","Data":"8ad5bceceb3ab3ec54885b41b04f990e83a1af7d55f045dc871c6bf57737bc96"} Jul 27 23:38:11.598377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.598312 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" event={"ID":"57ef9b32-ce47-481b-b23d-550c82bcc50c","Type":"ContainerStarted","Data":"41ba64a06a4093e4addc28b15e50c10ced49480bd22a6e4dacc04c4080d2ab19"} Jul 27 23:38:11.599403 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.599379 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4w58m" event={"ID":"e704052e-dd94-4bf3-b3a2-1589b7c3ab62","Type":"ContainerStarted","Data":"5aef37e996c99ac778e2c68f7c6baa657f022db20a95e47351a78b6cea4cf019"} Jul 27 23:38:11.600439 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.600419 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7j27" event={"ID":"2c884103-50ba-403b-89eb-66d666438a5c","Type":"ContainerStarted","Data":"9c00dec0c2590a22e6237eb35effa54e6f299851a4c85350016c021d00fb4573"} Jul 27 23:38:11.611892 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.611830 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nkcxr" podStartSLOduration=2.605257998 podStartE2EDuration="3.611815495s" totalImagesPullingTime="933.005354ms" totalInitContainerRuntime="73.552143ms" isStatefulPod=true podCreationTimestamp="2026-07-27 23:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:11.611008221 +0000 UTC m=+66.971544901" watchObservedRunningTime="2026-07-27 23:38:11.611815495 +0000 UTC m=+66.972352150" Jul 27 23:38:11.627181 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.627132 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-94hps" podStartSLOduration=2.17035418 podStartE2EDuration="3.62711582s" totalImagesPullingTime="1.45676164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:11.626211373 +0000 UTC m=+66.986748050" watchObservedRunningTime="2026-07-27 23:38:11.62711582 +0000 UTC m=+66.987652553" Jul 27 23:38:11.740647 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:11.740607 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64d66f84d-2s9vf"] Jul 27 23:38:11.753591 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:11.753558 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5195fb0_43a0_4373_a1f9_68a8a504d511.slice/crio-87dfac6245f79f36c6067ec69bfc3848ad8b8bcfcfa5789e274bbaa99d3694a8 WatchSource:0}: Error finding container 87dfac6245f79f36c6067ec69bfc3848ad8b8bcfcfa5789e274bbaa99d3694a8: Status 404 returned error can't find the container with id 87dfac6245f79f36c6067ec69bfc3848ad8b8bcfcfa5789e274bbaa99d3694a8 Jul 27 23:38:12.594403 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.592956 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-55669fdbd7-f7wmg"] Jul 27 23:38:12.596769 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.595695 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.599362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.598853 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 27 23:38:12.599362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.599085 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 27 23:38:12.602382 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.600047 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 27 23:38:12.602382 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.600281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4sq4lstlgdr6t\"" Jul 27 23:38:12.602382 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.600445 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vkfb8\"" Jul 27 23:38:12.602382 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.600712 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 27 23:38:12.611937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.611208 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55669fdbd7-f7wmg"] Jul 27 23:38:12.613730 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.613452 2574 generic.go:356] "Generic (PLEG): container finished" podID="beb23daf-dbc2-49b5-9767-675995599626" containerID="dd86830d84b3b22ef4a30db756525e09032607ee587cfe9cff0f3b4c5035ea0b" exitCode=0 Jul 27 23:38:12.613935 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.613914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerDied","Data":"dd86830d84b3b22ef4a30db756525e09032607ee587cfe9cff0f3b4c5035ea0b"} Jul 27 23:38:12.618625 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.617655 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"87dfac6245f79f36c6067ec69bfc3848ad8b8bcfcfa5789e274bbaa99d3694a8"} Jul 27 23:38:12.653981 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.653934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-client-certs\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.653981 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.653982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.654249 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.654080 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da868074-54b4-4793-bc8e-84734837af1c-audit-log\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.654249 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.654127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-metrics-server-audit-profiles\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.654249 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.654164 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-tls\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.654249 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.654192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmd79\" (UniqueName: \"kubernetes.io/projected/da868074-54b4-4793-bc8e-84734837af1c-kube-api-access-tmd79\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.654489 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.654319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-client-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-client-certs\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755721 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755808 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da868074-54b4-4793-bc8e-84734837af1c-audit-log\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755839 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-metrics-server-audit-profiles\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755895 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-tls\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.755923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmd79\" (UniqueName: \"kubernetes.io/projected/da868074-54b4-4793-bc8e-84734837af1c-kube-api-access-tmd79\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.756765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.756017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-client-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.757919 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.757868 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da868074-54b4-4793-bc8e-84734837af1c-audit-log\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.758792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.758268 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-metrics-server-audit-profiles\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.758792 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.758755 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da868074-54b4-4793-bc8e-84734837af1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.761069 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.761023 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-client-certs\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.761162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.761086 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-secret-metrics-server-tls\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.761590 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.761566 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da868074-54b4-4793-bc8e-84734837af1c-client-ca-bundle\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.764441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.764416 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmd79\" (UniqueName: \"kubernetes.io/projected/da868074-54b4-4793-bc8e-84734837af1c-kube-api-access-tmd79\") pod \"metrics-server-55669fdbd7-f7wmg\" (UID: \"da868074-54b4-4793-bc8e-84734837af1c\") " pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.919824 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.919792 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:12.971838 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.971676 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:38:12.976562 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.975255 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm"] Jul 27 23:38:12.981433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.981410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:12.986461 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.986030 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 27 23:38:12.986461 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.986191 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm"] Jul 27 23:38:12.986461 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:12.986300 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-65vlg\"" Jul 27 23:38:13.006125 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.006100 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:38:13.010961 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.010938 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.019713 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.019655 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq7bg\" (UniqueName: \"kubernetes.io/projected/bacf92b3-e95d-4309-9b5a-3dd638203d08-kube-api-access-wq7bg\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060353 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qst8f\" (UniqueName: \"kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060428 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060470 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.060775 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.060504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.129054 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.128980 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55669fdbd7-f7wmg"] Jul 27 23:38:13.136103 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:13.136068 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda868074_54b4_4793_bc8e_84734837af1c.slice/crio-abb90814300da95cda0ae671346757dff6aa1ca06fc6518ba88739ee85776d0c WatchSource:0}: Error finding container abb90814300da95cda0ae671346757dff6aa1ca06fc6518ba88739ee85776d0c: Status 404 returned error can't find the container with id abb90814300da95cda0ae671346757dff6aa1ca06fc6518ba88739ee85776d0c Jul 27 23:38:13.161362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161377 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161416 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161446 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq7bg\" (UniqueName: \"kubernetes.io/projected/bacf92b3-e95d-4309-9b5a-3dd638203d08-kube-api-access-wq7bg\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161791 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.161823 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qst8f\" (UniqueName: \"kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:13.162507 2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 27 23:38:13.162937 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:13.162583 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert podName:bacf92b3-e95d-4309-9b5a-3dd638203d08 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:13.662567506 +0000 UTC m=+69.023104164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-2zgqm" (UID: "bacf92b3-e95d-4309-9b5a-3dd638203d08") : secret "monitoring-plugin-cert" not found Jul 27 23:38:13.172337 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.165312 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.172337 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.166355 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.173141 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.172752 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.173141 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.173103 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.173323 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.173301 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.175954 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.175875 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.176586 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.176567 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qst8f\" (UniqueName: \"kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f\") pod \"console-7b66684575-td5xz\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.177687 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.177660 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq7bg\" (UniqueName: \"kubernetes.io/projected/bacf92b3-e95d-4309-9b5a-3dd638203d08-kube-api-access-wq7bg\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.344609 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.344569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:13.536066 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.536039 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:38:13.627404 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.627362 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4w58m" event={"ID":"e704052e-dd94-4bf3-b3a2-1589b7c3ab62","Type":"ContainerStarted","Data":"6d341ceb7a5be824cb3284d62fc81f9f932dce1ca795bc06077edf9ef57db414"} Jul 27 23:38:13.627874 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.627426 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4w58m" event={"ID":"e704052e-dd94-4bf3-b3a2-1589b7c3ab62","Type":"ContainerStarted","Data":"fcf4f4d419876086702e738c8f44abe2d98ce43f4a52d89af4d57202c1709cdd"} Jul 27 23:38:13.630599 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.630484 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" event={"ID":"da868074-54b4-4793-bc8e-84734837af1c","Type":"ContainerStarted","Data":"abb90814300da95cda0ae671346757dff6aa1ca06fc6518ba88739ee85776d0c"} Jul 27 23:38:13.643843 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.643187 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4w58m" podStartSLOduration=67.445338325 podStartE2EDuration="1m8.643167777s" totalImagesPullingTime="1.197829452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:13.642597289 +0000 UTC m=+69.003133967" watchObservedRunningTime="2026-07-27 23:38:13.643167777 +0000 UTC m=+69.003704456" Jul 27 23:38:13.670388 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.670338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.674221 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.674166 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bacf92b3-e95d-4309-9b5a-3dd638203d08-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2zgqm\" (UID: \"bacf92b3-e95d-4309-9b5a-3dd638203d08\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:13.922621 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:13.922583 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:14.362360 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:14.362287 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25612a1f_c19f_49b0_84fc_79b4acdbe8ef.slice/crio-6f80665fca97936661dd3e66b66bc1085629fbcdc60087856147b50e91eb8a7d WatchSource:0}: Error finding container 6f80665fca97936661dd3e66b66bc1085629fbcdc60087856147b50e91eb8a7d: Status 404 returned error can't find the container with id 6f80665fca97936661dd3e66b66bc1085629fbcdc60087856147b50e91eb8a7d Jul 27 23:38:14.451481 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.450794 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 27 23:38:14.469794 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.469761 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.474366 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.474848 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.475054 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.476342 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.476554 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.476762 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2f0plehvqu3mk\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.476928 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.477120 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 27 23:38:14.478780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.478599 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 27 23:38:14.479265 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.478879 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 27 23:38:14.479265 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.479113 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 27 23:38:14.479365 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.479305 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 27 23:38:14.479568 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.479507 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 27 23:38:14.480014 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.479983 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 27 23:38:14.481429 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.481407 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 27 23:38:14.481805 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.481772 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nlsql\"" Jul 27 23:38:14.484558 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.484530 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 27 23:38:14.581673 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581673 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581918 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581918 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581731 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581918 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581766 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581918 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581819 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.581918 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbdl\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-kube-api-access-xmbdl\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.581990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582029 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582062 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582083 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582170 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582220 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582250 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.582630 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.582427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.638890 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.638847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b66684575-td5xz" event={"ID":"25612a1f-c19f-49b0-84fc-79b4acdbe8ef","Type":"ContainerStarted","Data":"6f80665fca97936661dd3e66b66bc1085629fbcdc60087856147b50e91eb8a7d"} Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.682927 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.682979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbdl\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-kube-api-access-xmbdl\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683136 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683161 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683264 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683408 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.683726 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683448 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.684580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683468 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.684580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.684580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.683536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.684580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.684011 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.684580 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.684575 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.685760 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.685411 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.685760 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.685505 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.685974 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.685943 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.687021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.686435 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.687021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.686965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.688661 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.687764 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.688661 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.688206 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.688661 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.688601 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.689197 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689145 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a584638-ca00-4381-a778-649abf5141b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689706 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689799 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a584638-ca00-4381-a778-649abf5141b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689856 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689929 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.689977 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.690888 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.690842 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.691272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.691250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.692350 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.692328 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a584638-ca00-4381-a778-649abf5141b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.692829 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.692793 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbdl\" (UniqueName: \"kubernetes.io/projected/6a584638-ca00-4381-a778-649abf5141b2-kube-api-access-xmbdl\") pod \"prometheus-k8s-0\" (UID: \"6a584638-ca00-4381-a778-649abf5141b2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:14.792967 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:14.792895 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:17.423754 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:17.423624 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm"] Jul 27 23:38:17.458717 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:17.458692 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 27 23:38:17.703363 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:17.703318 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a584638_ca00_4381_a778_649abf5141b2.slice/crio-4e0f940782c35ddc30aae72cb615ec149e8de64304631c73b9ad28c3ac563c48 WatchSource:0}: Error finding container 4e0f940782c35ddc30aae72cb615ec149e8de64304631c73b9ad28c3ac563c48: Status 404 returned error can't find the container with id 4e0f940782c35ddc30aae72cb615ec149e8de64304631c73b9ad28c3ac563c48 Jul 27 23:38:17.705950 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:38:17.705917 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacf92b3_e95d_4309_9b5a_3dd638203d08.slice/crio-c45861b13e6774706dbcd60660b7cd7c2f15d0d52b8ca4e5922f6df506a18397 WatchSource:0}: Error finding container c45861b13e6774706dbcd60660b7cd7c2f15d0d52b8ca4e5922f6df506a18397: Status 404 returned error can't find the container with id c45861b13e6774706dbcd60660b7cd7c2f15d0d52b8ca4e5922f6df506a18397 Jul 27 23:38:18.661623 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.661585 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nr6fd" event={"ID":"2067dc2d-166c-4f47-aae9-87373917a88f","Type":"ContainerStarted","Data":"a1cbc2df8118c97ca32b664ba6fa87518f1e0413272f5bc8b0725d4992a17c80"} Jul 27 23:38:18.662205 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.662177 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:38:18.664075 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.664051 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7j27" event={"ID":"2c884103-50ba-403b-89eb-66d666438a5c","Type":"ContainerStarted","Data":"0ba8a4a525292c258a75633c03615eabb5ec587a49170eb7f0d667b94d704797"} Jul 27 23:38:18.665751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.665691 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" event={"ID":"bacf92b3-e95d-4309-9b5a-3dd638203d08","Type":"ContainerStarted","Data":"c45861b13e6774706dbcd60660b7cd7c2f15d0d52b8ca4e5922f6df506a18397"} Jul 27 23:38:18.667490 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.667464 2574 generic.go:356] "Generic (PLEG): container finished" podID="6a584638-ca00-4381-a778-649abf5141b2" containerID="83f6c5879833291f6a7d656410b7d4c75e6e2a27ae6237999c56728ce480e27b" exitCode=0 Jul 27 23:38:18.667652 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.667567 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerDied","Data":"83f6c5879833291f6a7d656410b7d4c75e6e2a27ae6237999c56728ce480e27b"} Jul 27 23:38:18.667652 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.667643 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"4e0f940782c35ddc30aae72cb615ec149e8de64304631c73b9ad28c3ac563c48"} Jul 27 23:38:18.670496 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.670463 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"6c9fce0adb1f8dbb0cee42e96978e56c629527e382b77d28d7f8f2547e0c37ef"} Jul 27 23:38:18.670640 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.670534 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"33abbd4ce2c02de51441d5ec501f10978f45f8d3a0f1d6fc825fef2d99313da9"} Jul 27 23:38:18.670640 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.670575 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"ba3ca016c0f8c15f0e39db29001cf29a4fb3302f4face1bd4b683bdf1d29b484"} Jul 27 23:38:18.672247 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.672225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" event={"ID":"da868074-54b4-4793-bc8e-84734837af1c","Type":"ContainerStarted","Data":"f1465d65971167f0b3477a1ea6a39e419bb238cdac1d2c216cd12ffd33f0b4ba"} Jul 27 23:38:18.673780 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.673755 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b66684575-td5xz" event={"ID":"25612a1f-c19f-49b0-84fc-79b4acdbe8ef","Type":"ContainerStarted","Data":"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66"} Jul 27 23:38:18.677320 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.677299 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"2381514cdf10089a6b3965b6a703a05566fb5ce888de5a116fa3164605518e91"} Jul 27 23:38:18.677425 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.677359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"324870a37772476248cbbe8cda879122a8a77165aa9dccb6e1094e246b263071"} Jul 27 23:38:18.677425 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.677391 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"6e70b9cf419030f60ff0fe9550950f738f47ea753cfa50093b84280a8ef6793f"} Jul 27 23:38:18.677425 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.677416 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"efef2d4ca9e4e33abafcf87ea0b77dd85296b3e49a9cf496bd494934d387811b"} Jul 27 23:38:18.677597 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.677448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"d0d4deb9813232230dbd3c73f85320711632734c2f54aa52685a9bca7e0318b9"} Jul 27 23:38:18.678124 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.678081 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nr6fd" podStartSLOduration=67.555448636 podStartE2EDuration="1m13.678067214s" totalImagesPullingTime="6.122618578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:18.676591556 +0000 UTC m=+74.037128234" watchObservedRunningTime="2026-07-27 23:38:18.678067214 +0000 UTC m=+74.038603891" Jul 27 23:38:18.693718 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.693666 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x7j27" podStartSLOduration=67.233506263 podStartE2EDuration="1m13.693651126s" totalImagesPullingTime="6.460144863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:18.692508473 +0000 UTC m=+74.053045152" watchObservedRunningTime="2026-07-27 23:38:18.693651126 +0000 UTC m=+74.054187853" Jul 27 23:38:18.711149 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.711082 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b66684575-td5xz" podStartSLOduration=6.711061883 podStartE2EDuration="6.711061883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:18.710944727 +0000 UTC m=+74.071481403" watchObservedRunningTime="2026-07-27 23:38:18.711061883 +0000 UTC m=+74.071598564" Jul 27 23:38:18.727809 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:18.727761 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" podStartSLOduration=2.106953418 podStartE2EDuration="6.727741266s" totalImagesPullingTime="4.620787848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:18.726715056 +0000 UTC m=+74.087251737" watchObservedRunningTime="2026-07-27 23:38:18.727741266 +0000 UTC m=+74.088277946" Jul 27 23:38:19.685477 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.685431 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"9098c43b0bf0f038fff5e6d5e29737c6a21677a57fb6d5f8342c8e59a9fea3df"} Jul 27 23:38:19.685965 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.685548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"b141675c5e95e889e944ad67b3ca329c8320112473440e24c72f6d041643709a"} Jul 27 23:38:19.685965 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.685592 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" event={"ID":"b5195fb0-43a0-4373-a1f9-68a8a504d511","Type":"ContainerStarted","Data":"922ea26d2e40792653a25fe35e1e7cfdbd87e9b0655f86ba70df74324da7f6e4"} Jul 27 23:38:19.686994 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.686958 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:19.691670 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.691621 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beb23daf-dbc2-49b5-9767-675995599626","Type":"ContainerStarted","Data":"69f26d0f6885c3faf02de1fb76d8cb8a0ace7f90d3b346fbc3439ad18a238fc5"} Jul 27 23:38:19.693405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.693376 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" event={"ID":"bacf92b3-e95d-4309-9b5a-3dd638203d08","Type":"ContainerStarted","Data":"f84fb2d039e57c36eb396e35605dbd6db6103bc5af990a1bd02c5ba8e8b9b6fc"} Jul 27 23:38:19.710994 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.710941 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" podStartSLOduration=1.9026573930000001 podStartE2EDuration="8.710926126s" totalImagesPullingTime="6.808268733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:19.708622013 +0000 UTC m=+75.069158691" watchObservedRunningTime="2026-07-27 23:38:19.710926126 +0000 UTC m=+75.071462803" Jul 27 23:38:19.723738 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.723681 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" podStartSLOduration=6.157187466 podStartE2EDuration="7.723661844s" totalImagesPullingTime="1.566474378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:19.722164986 +0000 UTC m=+75.082701665" watchObservedRunningTime="2026-07-27 23:38:19.723661844 +0000 UTC m=+75.084198523" Jul 27 23:38:19.746235 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:19.746181 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.676572192 podStartE2EDuration="10.746159019s" totalImagesPullingTime="7.050956049s" totalInitContainerRuntime="18.630778ms" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:19.745155126 +0000 UTC m=+75.105691806" watchObservedRunningTime="2026-07-27 23:38:19.746159019 +0000 UTC m=+75.106695700" Jul 27 23:38:20.698674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:20.698594 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:20.705020 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:20.704995 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2zgqm" Jul 27 23:38:20.888910 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:20.888878 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:20.893345 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:20.893318 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:22.710546 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"50af1626ff253e8c399bd53f1e066ca94a43e29d98e7ed578e2542513472ab4e"} Jul 27 23:38:22.710942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710606 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"8f575b28c4dec3b4bd65e017dd1fa90409bfec85bc5cf2af703fea85d32a8d41"} Jul 27 23:38:22.710942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710632 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"38a818da63de576dd6e11707c4169b15906d8a216b4530fa2c8cf9cade232cbb"} Jul 27 23:38:22.710942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710658 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"920bb637dd63b9c101501f34e418bbfc87f17e17e50bb66632ec09f59d167ae4"} Jul 27 23:38:22.710942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710682 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"0174c831e4132a388b93e16c8ac677c7d7a46dcf2bfefe645cab89ce4aa5f911"} Jul 27 23:38:22.710942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.710706 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a584638-ca00-4381-a778-649abf5141b2","Type":"ContainerStarted","Data":"c7bb8b5b2fad1dd3997b0bfad0cc533c749cb5fec5218b4c0773353d89f391bf"} Jul 27 23:38:22.736122 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:22.736058 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.426896949 podStartE2EDuration="8.7360364s" totalImagesPullingTime="3.272454412s" totalInitContainerRuntime="36.685039ms" isStatefulPod=false podCreationTimestamp="2026-07-27 23:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:22.734731725 +0000 UTC m=+78.095268405" watchObservedRunningTime="2026-07-27 23:38:22.7360364 +0000 UTC m=+78.096573079" Jul 27 23:38:23.345703 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:23.345660 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:23.345880 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:23.345739 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:23.350414 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:23.350393 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:23.719565 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:23.719540 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:38:23.761360 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:23.761333 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:24.794001 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:24.793955 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:38:26.710165 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:26.710047 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64d66f84d-2s9vf" Jul 27 23:38:32.920601 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:32.920567 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:32.921036 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:32.920652 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:35.462319 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:35.462289 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4xlw5_6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c/serve-healthcheck-canary/0.log" Jul 27 23:38:38.004301 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.004242 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dc8c94bd4-tpzrb" podUID="8b546693-11ae-4e87-87d6-556009964b26" containerName="console" containerID="cri-o://c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f" gracePeriod=15 Jul 27 23:38:38.779898 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.779860 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:38:38.785467 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.785438 2574 generic.go:356] "Generic (PLEG): container finished" podID="8b546693-11ae-4e87-87d6-556009964b26" containerID="c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f" exitCode=0 Jul 27 23:38:38.785682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.785490 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8c94bd4-tpzrb" event={"ID":"8b546693-11ae-4e87-87d6-556009964b26","Type":"ContainerDied","Data":"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f"} Jul 27 23:38:38.785682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.785496 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8c94bd4-tpzrb" Jul 27 23:38:38.785682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.785543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8c94bd4-tpzrb" event={"ID":"8b546693-11ae-4e87-87d6-556009964b26","Type":"ContainerDied","Data":"65be2c6b5bfbf7a08e3dd7d450b87d9edfbab4d1728f5b3df309c215524b388e"} Jul 27 23:38:38.785682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.785567 2574 scope.go:117] "RemoveContainer" containerID="c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f" Jul 27 23:38:38.793903 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.793881 2574 scope.go:117] "RemoveContainer" containerID="c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f" Jul 27 23:38:38.794166 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:38.794147 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f\": container with ID starting with c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f not found: ID does not exist" containerID="c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f" Jul 27 23:38:38.794210 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.794179 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f"} err="failed to get container status \"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f\": rpc error: code = NotFound desc = could not find container \"c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f\": container with ID starting with c40f0849d18d03fa768a64c6ca017ab417491e7630df43e9e1b9d206dacec95f not found: ID does not exist" Jul 27 23:38:38.815845 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.815809 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.815913 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.815943 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlpk2\" (UniqueName: \"kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816021 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.815996 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816178 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.816026 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816178 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.816060 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert\") pod \"8b546693-11ae-4e87-87d6-556009964b26\" (UID: \"8b546693-11ae-4e87-87d6-556009964b26\") " Jul 27 23:38:38.816379 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.816350 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca" (OuterVolumeSpecName: "service-ca") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:38.816476 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.816454 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:38.816552 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.816481 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config" (OuterVolumeSpecName: "console-config") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:38.818397 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.818355 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:38.818545 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.818460 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2" (OuterVolumeSpecName: "kube-api-access-jlpk2") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "kube-api-access-jlpk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:38:38.818545 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.818464 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8b546693-11ae-4e87-87d6-556009964b26" (UID: "8b546693-11ae-4e87-87d6-556009964b26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:38.917079 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917041 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-console-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:38.917079 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917071 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:38.917079 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917081 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-service-ca\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:38.917079 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917089 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b546693-11ae-4e87-87d6-556009964b26-console-oauth-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:38.917347 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917098 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlpk2\" (UniqueName: \"kubernetes.io/projected/8b546693-11ae-4e87-87d6-556009964b26-kube-api-access-jlpk2\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:38.917347 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:38.917106 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b546693-11ae-4e87-87d6-556009964b26-oauth-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:39.107757 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:39.107705 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:38:39.111215 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:39.111187 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dc8c94bd4-tpzrb"] Jul 27 23:38:39.172852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:39.172818 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b546693-11ae-4e87-87d6-556009964b26" path="/var/lib/kubelet/pods/8b546693-11ae-4e87-87d6-556009964b26/volumes" Jul 27 23:38:48.781253 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:48.781191 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-db94477dd-phmlm" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerName="console" containerID="cri-o://b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811" gracePeriod=15 Jul 27 23:38:49.067765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.067732 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.112947 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113001 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113033 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl2n7\" (UniqueName: \"kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113065 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113130 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113211 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.113846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.113240 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert\") pod \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\" (UID: \"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f\") " Jul 27 23:38:49.115567 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.115005 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:49.115567 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.115265 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:49.115567 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.115291 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:49.116130 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.116078 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config" (OuterVolumeSpecName: "console-config") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:49.121931 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.121890 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:49.122072 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.122049 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7" (OuterVolumeSpecName: "kube-api-access-bl2n7") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "kube-api-access-bl2n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:38:49.122294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.122270 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" (UID: "4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214884 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-service-ca\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214924 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214940 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-oauth-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214955 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-console-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214971 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl2n7\" (UniqueName: \"kubernetes.io/projected/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-kube-api-access-bl2n7\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.214985 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-oauth-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.215030 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.215001 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f-trusted-ca-bundle\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:38:49.839419 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.839381 2574 generic.go:356] "Generic (PLEG): container finished" podID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerID="b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811" exitCode=0 Jul 27 23:38:49.839852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.839436 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db94477dd-phmlm" Jul 27 23:38:49.839852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.839468 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db94477dd-phmlm" event={"ID":"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f","Type":"ContainerDied","Data":"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811"} Jul 27 23:38:49.839852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.839547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db94477dd-phmlm" event={"ID":"4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f","Type":"ContainerDied","Data":"e62a4c3b3ad38cf9ac7c582210ecc744f834552f3efc1aab4abd9cee0ef429d3"} Jul 27 23:38:49.839852 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.839575 2574 scope.go:117] "RemoveContainer" containerID="b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811" Jul 27 23:38:49.848666 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.848637 2574 scope.go:117] "RemoveContainer" containerID="b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811" Jul 27 23:38:49.849186 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:38:49.849108 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811\": container with ID starting with b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811 not found: ID does not exist" containerID="b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811" Jul 27 23:38:49.849186 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.849143 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811"} err="failed to get container status \"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811\": rpc error: code = NotFound desc = could not find container \"b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811\": container with ID starting with b3a49159df35425fb2939b2f656425b9ccd93a37b8e7a3df1d79abd49cbcc811 not found: ID does not exist" Jul 27 23:38:49.856751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.856723 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:49.862655 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:49.862629 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-db94477dd-phmlm"] Jul 27 23:38:50.701711 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:50.701673 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nr6fd" Jul 27 23:38:51.172583 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:51.172550 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" path="/var/lib/kubelet/pods/4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f/volumes" Jul 27 23:38:52.925308 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:52.925273 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:38:52.928977 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:38:52.928954 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-55669fdbd7-f7wmg" Jul 27 23:39:14.793724 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:14.793664 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:39:14.813145 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:14.813119 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:39:14.958196 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:14.958169 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 27 23:39:37.749577 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.749534 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:39:37.750068 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750054 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b546693-11ae-4e87-87d6-556009964b26" containerName="console" Jul 27 23:39:37.750117 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750074 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b546693-11ae-4e87-87d6-556009964b26" containerName="console" Jul 27 23:39:37.750117 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750105 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerName="console" Jul 27 23:39:37.750117 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750113 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerName="console" Jul 27 23:39:37.750238 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750225 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b546693-11ae-4e87-87d6-556009964b26" containerName="console" Jul 27 23:39:37.750290 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.750240 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb7cd3c-6c0d-4047-9ccb-975cbd682a0f" containerName="console" Jul 27 23:39:37.752475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.752452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.763636 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.763607 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:39:37.852613 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852810 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852810 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852810 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852971 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852971 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xgh\" (UniqueName: \"kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.852971 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.852936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953691 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953705 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xgh\" (UniqueName: \"kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953782 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.953873 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953859 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.954151 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.953913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.955034 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.954996 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.955171 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.954996 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.955171 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.955039 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.955496 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.955470 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.956405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.956377 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.956480 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.956423 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:37.962240 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:37.962219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xgh\" (UniqueName: \"kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh\") pod \"console-5ccc8bc55-62ml4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:38.065428 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:38.065331 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:38.196183 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:38.196155 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:39:38.198760 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:39:38.198731 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78060bc5_ff28_42de_9039_062eb2d614a4.slice/crio-d423aea97e7819c9df0082048f58de7b75ff646be95fb320273236b6ae04bf97 WatchSource:0}: Error finding container d423aea97e7819c9df0082048f58de7b75ff646be95fb320273236b6ae04bf97: Status 404 returned error can't find the container with id d423aea97e7819c9df0082048f58de7b75ff646be95fb320273236b6ae04bf97 Jul 27 23:39:39.039223 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:39.039190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccc8bc55-62ml4" event={"ID":"78060bc5-ff28-42de-9039-062eb2d614a4","Type":"ContainerStarted","Data":"aebcfc55aa67f1cb7528edfe54dd06e871fde80c44ce44034dc4452167ecc117"} Jul 27 23:39:39.039625 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:39.039240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccc8bc55-62ml4" event={"ID":"78060bc5-ff28-42de-9039-062eb2d614a4","Type":"ContainerStarted","Data":"d423aea97e7819c9df0082048f58de7b75ff646be95fb320273236b6ae04bf97"} Jul 27 23:39:39.059846 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:39.059794 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ccc8bc55-62ml4" podStartSLOduration=2.059778813 podStartE2EDuration="2.059778813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:39:39.058286837 +0000 UTC m=+154.418823516" watchObservedRunningTime="2026-07-27 23:39:39.059778813 +0000 UTC m=+154.420315491" Jul 27 23:39:42.972156 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:42.972115 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:39:43.003958 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.003928 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68cffd5c54-4672h"] Jul 27 23:39:43.006302 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.006282 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.017985 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.017958 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68cffd5c54-4672h"] Jul 27 23:39:43.100215 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-trusted-ca-bundle\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100215 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100219 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-service-ca\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-oauth-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100359 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkp8j\" (UniqueName: \"kubernetes.io/projected/b64dc2c3-9b55-4d99-9485-3767eff15c0a-kube-api-access-lkp8j\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100390 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.100556 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.100431 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-oauth-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.201658 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-oauth-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.201658 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201658 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkp8j\" (UniqueName: \"kubernetes.io/projected/b64dc2c3-9b55-4d99-9485-3767eff15c0a-kube-api-access-lkp8j\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.201876 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.201876 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-oauth-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.201959 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-trusted-ca-bundle\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202018 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.201974 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202018 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.202010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-service-ca\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202461 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.202432 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-oauth-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202743 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.202717 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202832 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.202745 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-service-ca\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.202832 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.202788 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b64dc2c3-9b55-4d99-9485-3767eff15c0a-trusted-ca-bundle\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.204187 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.204169 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-oauth-config\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.204302 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.204284 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b64dc2c3-9b55-4d99-9485-3767eff15c0a-console-serving-cert\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.210713 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.210692 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkp8j\" (UniqueName: \"kubernetes.io/projected/b64dc2c3-9b55-4d99-9485-3767eff15c0a-kube-api-access-lkp8j\") pod \"console-68cffd5c54-4672h\" (UID: \"b64dc2c3-9b55-4d99-9485-3767eff15c0a\") " pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.315584 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.315470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:43.464432 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:43.464393 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68cffd5c54-4672h"] Jul 27 23:39:43.467361 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:39:43.467333 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64dc2c3_9b55_4d99_9485_3767eff15c0a.slice/crio-b63f0fcf066f3c82b1bf5baf9947cd8d05a1c0994557963af682c82f66a5d5e1 WatchSource:0}: Error finding container b63f0fcf066f3c82b1bf5baf9947cd8d05a1c0994557963af682c82f66a5d5e1: Status 404 returned error can't find the container with id b63f0fcf066f3c82b1bf5baf9947cd8d05a1c0994557963af682c82f66a5d5e1 Jul 27 23:39:44.060231 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:44.060197 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cffd5c54-4672h" event={"ID":"b64dc2c3-9b55-4d99-9485-3767eff15c0a","Type":"ContainerStarted","Data":"b648e7cca12b99707939ec2f27dce3cdb38e30332d65d6924f3c90a6188db580"} Jul 27 23:39:44.060626 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:44.060243 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cffd5c54-4672h" event={"ID":"b64dc2c3-9b55-4d99-9485-3767eff15c0a","Type":"ContainerStarted","Data":"b63f0fcf066f3c82b1bf5baf9947cd8d05a1c0994557963af682c82f66a5d5e1"} Jul 27 23:39:44.078059 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:44.078002 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68cffd5c54-4672h" podStartSLOduration=2.077986821 podStartE2EDuration="2.077986821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:39:44.07646529 +0000 UTC m=+159.437001980" watchObservedRunningTime="2026-07-27 23:39:44.077986821 +0000 UTC m=+159.438523498" Jul 27 23:39:48.065648 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:48.065595 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:39:53.316353 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:53.316312 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:53.316781 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:53.316381 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:53.321024 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:53.321002 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:54.108020 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:54.107990 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68cffd5c54-4672h" Jul 27 23:39:54.158316 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:39:54.158283 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:40:07.990747 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:07.990690 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5ccc8bc55-62ml4" podUID="78060bc5-ff28-42de-9039-062eb2d614a4" containerName="console" containerID="cri-o://aebcfc55aa67f1cb7528edfe54dd06e871fde80c44ce44034dc4452167ecc117" gracePeriod=15 Jul 27 23:40:08.160732 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.160606 2574 generic.go:356] "Generic (PLEG): container finished" podID="78060bc5-ff28-42de-9039-062eb2d614a4" containerID="aebcfc55aa67f1cb7528edfe54dd06e871fde80c44ce44034dc4452167ecc117" exitCode=0 Jul 27 23:40:08.160879 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.160774 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccc8bc55-62ml4" event={"ID":"78060bc5-ff28-42de-9039-062eb2d614a4","Type":"ContainerDied","Data":"aebcfc55aa67f1cb7528edfe54dd06e871fde80c44ce44034dc4452167ecc117"} Jul 27 23:40:08.227955 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.227932 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:40:08.316946 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.316854 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.316946 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.316903 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.316954 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.316974 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.316995 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7xgh\" (UniqueName: \"kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317021 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317047 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config\") pod \"78060bc5-ff28-42de-9039-062eb2d614a4\" (UID: \"78060bc5-ff28-42de-9039-062eb2d614a4\") " Jul 27 23:40:08.317457 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317425 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:08.317636 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317598 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config" (OuterVolumeSpecName: "console-config") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:08.317717 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317697 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:08.317717 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.317703 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca" (OuterVolumeSpecName: "service-ca") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:08.319253 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.319229 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:40:08.319349 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.319255 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh" (OuterVolumeSpecName: "kube-api-access-h7xgh") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "kube-api-access-h7xgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:40:08.319349 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.319307 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "78060bc5-ff28-42de-9039-062eb2d614a4" (UID: "78060bc5-ff28-42de-9039-062eb2d614a4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:40:08.418193 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418157 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-oauth-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418193 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418195 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78060bc5-ff28-42de-9039-062eb2d614a4-console-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418209 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7xgh\" (UniqueName: \"kubernetes.io/projected/78060bc5-ff28-42de-9039-062eb2d614a4-kube-api-access-h7xgh\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418225 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-service-ca\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418238 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-console-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418250 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-trusted-ca-bundle\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:08.418440 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:08.418263 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78060bc5-ff28-42de-9039-062eb2d614a4-oauth-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:09.166576 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:09.166546 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccc8bc55-62ml4" Jul 27 23:40:09.166942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:09.166546 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccc8bc55-62ml4" event={"ID":"78060bc5-ff28-42de-9039-062eb2d614a4","Type":"ContainerDied","Data":"d423aea97e7819c9df0082048f58de7b75ff646be95fb320273236b6ae04bf97"} Jul 27 23:40:09.166942 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:09.166751 2574 scope.go:117] "RemoveContainer" containerID="aebcfc55aa67f1cb7528edfe54dd06e871fde80c44ce44034dc4452167ecc117" Jul 27 23:40:09.188688 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:09.188625 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:40:09.190593 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:09.190562 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5ccc8bc55-62ml4"] Jul 27 23:40:11.172330 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:11.172296 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78060bc5-ff28-42de-9039-062eb2d614a4" path="/var/lib/kubelet/pods/78060bc5-ff28-42de-9039-062eb2d614a4/volumes" Jul 27 23:40:19.178122 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.178080 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b66684575-td5xz" podUID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" containerName="console" containerID="cri-o://98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66" gracePeriod=15 Jul 27 23:40:19.418626 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.418597 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:40:19.520363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520280 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520313 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520363 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520361 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520388 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qst8f\" (UniqueName: \"kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520434 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520476 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520674 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520504 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config\") pod \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\" (UID: \"25612a1f-c19f-49b0-84fc-79b4acdbe8ef\") " Jul 27 23:40:19.520874 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520744 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:19.520874 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520807 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:19.520994 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.520977 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-service-ca\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.521043 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.521003 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config" (OuterVolumeSpecName: "console-config") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:19.521043 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.521021 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:40:19.522702 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.522679 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:40:19.523135 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.523114 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:40:19.523205 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.523146 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f" (OuterVolumeSpecName: "kube-api-access-qst8f") pod "25612a1f-c19f-49b0-84fc-79b4acdbe8ef" (UID: "25612a1f-c19f-49b0-84fc-79b4acdbe8ef"). InnerVolumeSpecName "kube-api-access-qst8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622234 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-trusted-ca-bundle\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622281 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-oauth-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622290 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qst8f\" (UniqueName: \"kubernetes.io/projected/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-kube-api-access-qst8f\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622299 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-oauth-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622308 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-serving-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:19.622291 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:19.622317 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25612a1f-c19f-49b0-84fc-79b4acdbe8ef-console-config\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:40:20.214975 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.214942 2574 generic.go:356] "Generic (PLEG): container finished" podID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" containerID="98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66" exitCode=0 Jul 27 23:40:20.215377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.214986 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b66684575-td5xz" Jul 27 23:40:20.215377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.214987 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b66684575-td5xz" event={"ID":"25612a1f-c19f-49b0-84fc-79b4acdbe8ef","Type":"ContainerDied","Data":"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66"} Jul 27 23:40:20.215377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.215030 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b66684575-td5xz" event={"ID":"25612a1f-c19f-49b0-84fc-79b4acdbe8ef","Type":"ContainerDied","Data":"6f80665fca97936661dd3e66b66bc1085629fbcdc60087856147b50e91eb8a7d"} Jul 27 23:40:20.215377 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.215050 2574 scope.go:117] "RemoveContainer" containerID="98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66" Jul 27 23:40:20.227578 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.227555 2574 scope.go:117] "RemoveContainer" containerID="98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66" Jul 27 23:40:20.247680 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:40:20.247645 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66\": container with ID starting with 98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66 not found: ID does not exist" containerID="98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66" Jul 27 23:40:20.247856 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.247827 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66"} err="failed to get container status \"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66\": rpc error: code = NotFound desc = could not find container \"98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66\": container with ID starting with 98b924924ed140314ff861664fa3e6abd880ba0467e599b96da4e1324fd2ea66 not found: ID does not exist" Jul 27 23:40:20.264506 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.264480 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:40:20.266078 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:20.266059 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b66684575-td5xz"] Jul 27 23:40:21.172283 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:40:21.172249 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" path="/var/lib/kubelet/pods/25612a1f-c19f-49b0-84fc-79b4acdbe8ef/volumes" Jul 27 23:41:13.744696 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.744661 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgnqk"] Jul 27 23:41:13.745405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745380 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" containerName="console" Jul 27 23:41:13.745405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745407 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" containerName="console" Jul 27 23:41:13.745561 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745429 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78060bc5-ff28-42de-9039-062eb2d614a4" containerName="console" Jul 27 23:41:13.745561 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745437 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="78060bc5-ff28-42de-9039-062eb2d614a4" containerName="console" Jul 27 23:41:13.745561 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745554 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="25612a1f-c19f-49b0-84fc-79b4acdbe8ef" containerName="console" Jul 27 23:41:13.745717 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.745570 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="78060bc5-ff28-42de-9039-062eb2d614a4" containerName="console" Jul 27 23:41:13.749899 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.749876 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:13.753277 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.753255 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 27 23:41:13.754361 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.754342 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mdlvs\"" Jul 27 23:41:13.754474 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.754457 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 27 23:41:13.760336 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.760315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgnqk"] Jul 27 23:41:13.923185 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.923150 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:13.923368 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:13.923284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kns65\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-kube-api-access-kns65\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.023773 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.023670 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kns65\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-kube-api-access-kns65\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.023773 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.023718 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.032751 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.032726 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.032877 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.032829 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kns65\" (UniqueName: \"kubernetes.io/projected/ec5ea723-ad0c-4773-8c23-b9404b486593-kube-api-access-kns65\") pod \"cert-manager-webhook-6486ffd997-zgnqk\" (UID: \"ec5ea723-ad0c-4773-8c23-b9404b486593\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.067835 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.067796 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:14.195722 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.195693 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgnqk"] Jul 27 23:41:14.198964 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:41:14.198942 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec5ea723_ad0c_4773_8c23_b9404b486593.slice/crio-a051974e9c2acbf21f9ddb189d16b6a38da20115553fb0e12c7ffb80b4fcae63 WatchSource:0}: Error finding container a051974e9c2acbf21f9ddb189d16b6a38da20115553fb0e12c7ffb80b4fcae63: Status 404 returned error can't find the container with id a051974e9c2acbf21f9ddb189d16b6a38da20115553fb0e12c7ffb80b4fcae63 Jul 27 23:41:14.435966 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:14.435932 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" event={"ID":"ec5ea723-ad0c-4773-8c23-b9404b486593","Type":"ContainerStarted","Data":"a051974e9c2acbf21f9ddb189d16b6a38da20115553fb0e12c7ffb80b4fcae63"} Jul 27 23:41:17.451199 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:17.451159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" event={"ID":"ec5ea723-ad0c-4773-8c23-b9404b486593","Type":"ContainerStarted","Data":"1be3a233be3af1c7f8f5f3579026d6000002010664a423437f62b1e09336f597"} Jul 27 23:41:17.451681 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:17.451226 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:17.471257 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:17.471206 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" podStartSLOduration=1.652404367 podStartE2EDuration="4.471191471s" totalImagesPullingTime="2.818787104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:17.468955402 +0000 UTC m=+252.829492080" watchObservedRunningTime="2026-07-27 23:41:17.471191471 +0000 UTC m=+252.831728150" Jul 27 23:41:23.459815 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:23.459779 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zgnqk" Jul 27 23:41:25.756700 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.756661 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq"] Jul 27 23:41:25.763961 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.763937 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.767172 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.767143 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 27 23:41:25.768086 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.768065 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:41:25.768186 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.768082 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7wlwx\"" Jul 27 23:41:25.772301 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.772276 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq"] Jul 27 23:41:25.827460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.827420 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhpv\" (UniqueName: \"kubernetes.io/projected/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-kube-api-access-xjhpv\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.827699 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.827470 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.928212 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.928179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhpv\" (UniqueName: \"kubernetes.io/projected/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-kube-api-access-xjhpv\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.928212 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.928216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.928610 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.928593 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:25.938189 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:25.938164 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhpv\" (UniqueName: \"kubernetes.io/projected/82674fad-05a0-4b4c-ad8b-2bca7b8c4b73-kube-api-access-xjhpv\") pod \"openshift-lws-operator-fd8ccff4c-86fnq\" (UID: \"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:26.074558 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:26.074446 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" Jul 27 23:41:26.209463 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:26.209428 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq"] Jul 27 23:41:26.212560 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:41:26.212527 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82674fad_05a0_4b4c_ad8b_2bca7b8c4b73.slice/crio-21525a9296bc8faf30786e2e092b617ea801020db1aac8d6d02eba4f70ffb7b4 WatchSource:0}: Error finding container 21525a9296bc8faf30786e2e092b617ea801020db1aac8d6d02eba4f70ffb7b4: Status 404 returned error can't find the container with id 21525a9296bc8faf30786e2e092b617ea801020db1aac8d6d02eba4f70ffb7b4 Jul 27 23:41:26.491493 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:26.491455 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" event={"ID":"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73","Type":"ContainerStarted","Data":"21525a9296bc8faf30786e2e092b617ea801020db1aac8d6d02eba4f70ffb7b4"} Jul 27 23:41:30.510257 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:30.510220 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" event={"ID":"82674fad-05a0-4b4c-ad8b-2bca7b8c4b73","Type":"ContainerStarted","Data":"fe372311e74adbcc44f553607f0c7de73bbbdaf368986b51fce30f3676339ea6"} Jul 27 23:41:30.544781 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:30.544725 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-86fnq" podStartSLOduration=2.273976654 podStartE2EDuration="5.544710092s" totalImagesPullingTime="3.270733438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:30.543333562 +0000 UTC m=+265.903870242" watchObservedRunningTime="2026-07-27 23:41:30.544710092 +0000 UTC m=+265.905246771" Jul 27 23:41:45.377675 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.377623 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h"] Jul 27 23:41:45.381149 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.381132 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.383505 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.383481 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 27 23:41:45.383824 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.383807 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 27 23:41:45.384888 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.384868 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xmfgl\"" Jul 27 23:41:45.384986 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.384927 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 27 23:41:45.399530 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.396849 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h"] Jul 27 23:41:45.506657 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.506625 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.506657 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.506662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-manager-config\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.506884 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.506686 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-metrics-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.506884 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.506705 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49znx\" (UniqueName: \"kubernetes.io/projected/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-kube-api-access-49znx\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.607590 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.607553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.607590 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.607594 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-manager-config\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.607844 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.607616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-metrics-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.607844 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.607634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49znx\" (UniqueName: \"kubernetes.io/projected/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-kube-api-access-49znx\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.608334 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.608311 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-manager-config\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.610108 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.610086 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-metrics-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.610243 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.610222 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-cert\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.622055 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.622026 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49znx\" (UniqueName: \"kubernetes.io/projected/165bbeed-e12c-4ed2-b1bd-73c8ff78d64f-kube-api-access-49znx\") pod \"lws-controller-manager-65c89cb49-lcs4h\" (UID: \"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.691676 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.691625 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:45.836923 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:45.836895 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h"] Jul 27 23:41:45.839376 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:41:45.839345 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165bbeed_e12c_4ed2_b1bd_73c8ff78d64f.slice/crio-582dda8a90d4addbc886ce18e2298dbf738ac65c5e36d99f5881624bd824a181 WatchSource:0}: Error finding container 582dda8a90d4addbc886ce18e2298dbf738ac65c5e36d99f5881624bd824a181: Status 404 returned error can't find the container with id 582dda8a90d4addbc886ce18e2298dbf738ac65c5e36d99f5881624bd824a181 Jul 27 23:41:46.573793 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:46.573756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" event={"ID":"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f","Type":"ContainerStarted","Data":"582dda8a90d4addbc886ce18e2298dbf738ac65c5e36d99f5881624bd824a181"} Jul 27 23:41:48.583912 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.583817 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" event={"ID":"165bbeed-e12c-4ed2-b1bd-73c8ff78d64f","Type":"ContainerStarted","Data":"f1bf84ef1e6f68af34364e80106b9dedca3ef2403690d0a9ad9bc3cdb93f601d"} Jul 27 23:41:48.584283 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.583914 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:41:48.609102 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.609047 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" podStartSLOduration=1.296272603 podStartE2EDuration="3.609028061s" totalImagesPullingTime="2.312755458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:48.607783654 +0000 UTC m=+283.968320333" watchObservedRunningTime="2026-07-27 23:41:48.609028061 +0000 UTC m=+283.969564740" Jul 27 23:41:48.776988 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.776952 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6"] Jul 27 23:41:48.780465 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.780440 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.784378 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.784348 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 27 23:41:48.784821 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.784803 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 27 23:41:48.784907 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.784803 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 27 23:41:48.785313 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.785293 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rhhx7\"" Jul 27 23:41:48.786196 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.786177 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 27 23:41:48.811367 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.811333 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6"] Jul 27 23:41:48.835373 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.835275 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhc54\" (UniqueName: \"kubernetes.io/projected/51401b9e-3160-46e9-b807-9ee83bd55dbf-kube-api-access-nhc54\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.835373 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.835317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.835373 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.835351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.835648 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.835458 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.835648 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.835502 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.936412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhc54\" (UniqueName: \"kubernetes.io/projected/51401b9e-3160-46e9-b807-9ee83bd55dbf-kube-api-access-nhc54\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.936627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936420 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.936627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936448 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.936627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.936627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.937010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.936982 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.938986 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.938964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51401b9e-3160-46e9-b807-9ee83bd55dbf-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.939112 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.939092 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.939186 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.939170 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51401b9e-3160-46e9-b807-9ee83bd55dbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:48.945459 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:48.945434 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhc54\" (UniqueName: \"kubernetes.io/projected/51401b9e-3160-46e9-b807-9ee83bd55dbf-kube-api-access-nhc54\") pod \"opendatahub-operator-controller-manager-79b58bb86f-4mvj6\" (UID: \"51401b9e-3160-46e9-b807-9ee83bd55dbf\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:49.090283 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:49.090192 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:49.235413 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:49.235383 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6"] Jul 27 23:41:49.238156 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:41:49.238127 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51401b9e_3160_46e9_b807_9ee83bd55dbf.slice/crio-aa63710966e4f049665ed6e042b6cd1ec229e87d7ad162578860f3532c038e29 WatchSource:0}: Error finding container aa63710966e4f049665ed6e042b6cd1ec229e87d7ad162578860f3532c038e29: Status 404 returned error can't find the container with id aa63710966e4f049665ed6e042b6cd1ec229e87d7ad162578860f3532c038e29 Jul 27 23:41:49.589053 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:49.589017 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" event={"ID":"51401b9e-3160-46e9-b807-9ee83bd55dbf","Type":"ContainerStarted","Data":"aa63710966e4f049665ed6e042b6cd1ec229e87d7ad162578860f3532c038e29"} Jul 27 23:41:52.606277 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:52.606179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" event={"ID":"51401b9e-3160-46e9-b807-9ee83bd55dbf","Type":"ContainerStarted","Data":"8c329589ffb7022fc305b13ab91f35da242b54352afaf1dac97896f4cf75e311"} Jul 27 23:41:53.611817 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:53.611730 2574 generic.go:356] "Generic (PLEG): container finished" podID="51401b9e-3160-46e9-b807-9ee83bd55dbf" containerID="8c329589ffb7022fc305b13ab91f35da242b54352afaf1dac97896f4cf75e311" exitCode=0 Jul 27 23:41:53.611817 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:53.611770 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" event={"ID":"51401b9e-3160-46e9-b807-9ee83bd55dbf","Type":"ContainerDied","Data":"8c329589ffb7022fc305b13ab91f35da242b54352afaf1dac97896f4cf75e311"} Jul 27 23:41:54.618525 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:54.618472 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" event={"ID":"51401b9e-3160-46e9-b807-9ee83bd55dbf","Type":"ContainerStarted","Data":"03d447a9fbd820979da7dca907d497bc4d2f26eedbb47e5e47ba902201265e7f"} Jul 27 23:41:54.618897 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:54.618661 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:41:54.644782 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:54.644729 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" podStartSLOduration=2.349620126 podStartE2EDuration="6.644713909s" totalImagesPullingTime="3.281251118s" totalInitContainerRuntime="1.013842665s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:54.643965117 +0000 UTC m=+290.004501806" watchObservedRunningTime="2026-07-27 23:41:54.644713909 +0000 UTC m=+290.005250586" Jul 27 23:41:59.593155 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:41:59.593121 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-lcs4h" Jul 27 23:42:05.010459 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:05.010428 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:42:05.013888 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:05.013862 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 23:42:05.018441 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:05.018420 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:42:05.624644 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:05.624611 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-4mvj6" Jul 27 23:42:09.296259 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.296221 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f44c56477-4xj82"] Jul 27 23:42:09.299436 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.299419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.301946 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.301921 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 27 23:42:09.302287 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.302262 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 27 23:42:09.302541 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.302488 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 27 23:42:09.302541 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.302505 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 27 23:42:09.303353 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.303336 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jcb5t\"" Jul 27 23:42:09.309702 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.309680 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f44c56477-4xj82"] Jul 27 23:42:09.320977 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.320955 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbhsn\" (UniqueName: \"kubernetes.io/projected/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-kube-api-access-vbhsn\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.321080 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.320999 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tls-certs\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.321080 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.321049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tmp\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.421442 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.421408 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tmp\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.421660 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.421486 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbhsn\" (UniqueName: \"kubernetes.io/projected/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-kube-api-access-vbhsn\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.421660 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.421539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tls-certs\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.423929 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.423900 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tmp\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.424102 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.424084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-tls-certs\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.435774 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.435746 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbhsn\" (UniqueName: \"kubernetes.io/projected/57d2bec0-e711-4f92-bbff-9c1a8ecb8da7-kube-api-access-vbhsn\") pod \"kube-auth-proxy-6f44c56477-4xj82\" (UID: \"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7\") " pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.611025 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.610922 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" Jul 27 23:42:09.762816 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.762785 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f44c56477-4xj82"] Jul 27 23:42:09.765756 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:42:09.765727 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d2bec0_e711_4f92_bbff_9c1a8ecb8da7.slice/crio-be1b88fb607157346aa3ef85172ed0699196ade17a3d1b00ba3416b0794d986f WatchSource:0}: Error finding container be1b88fb607157346aa3ef85172ed0699196ade17a3d1b00ba3416b0794d986f: Status 404 returned error can't find the container with id be1b88fb607157346aa3ef85172ed0699196ade17a3d1b00ba3416b0794d986f Jul 27 23:42:09.767607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:09.767590 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:42:10.693627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:10.693305 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" event={"ID":"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7","Type":"ContainerStarted","Data":"be1b88fb607157346aa3ef85172ed0699196ade17a3d1b00ba3416b0794d986f"} Jul 27 23:42:12.704261 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:12.704172 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" event={"ID":"57d2bec0-e711-4f92-bbff-9c1a8ecb8da7","Type":"ContainerStarted","Data":"2ec668cf1c6b248dd78d9bd3f8def5cef2e0e2843d66efcb360e9025df26aad1"} Jul 27 23:42:12.720709 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:42:12.720656 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f44c56477-4xj82" podStartSLOduration=1.079480799 podStartE2EDuration="3.720640421s" totalImagesPullingTime="2.641159622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:42:12.719208141 +0000 UTC m=+308.079744819" watchObservedRunningTime="2026-07-27 23:42:12.720640421 +0000 UTC m=+308.081177151" Jul 27 23:43:55.919331 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.919297 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk"] Jul 27 23:43:55.922629 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.922612 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:55.926057 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.926039 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 27 23:43:55.926362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.926345 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 27 23:43:55.926497 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.926479 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 27 23:43:55.927138 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.927119 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 27 23:43:55.927362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.927344 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hwkjm\"" Jul 27 23:43:55.944463 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.944440 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk"] Jul 27 23:43:55.961919 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.961889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:55.962037 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.961991 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:55.962037 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:55.962018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8px\" (UniqueName: \"kubernetes.io/projected/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-kube-api-access-7h8px\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.063135 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.063103 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.063298 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.063137 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8px\" (UniqueName: \"kubernetes.io/projected/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-kube-api-access-7h8px\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.063298 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.063182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.063298 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:43:56.063261 2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 27 23:43:56.063454 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:43:56.063341 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert podName:cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b nodeName:}" failed. No retries permitted until 2026-07-27 23:43:56.56331859 +0000 UTC m=+411.923855247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-nkhnk" (UID: "cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b") : secret "plugin-serving-cert" not found Jul 27 23:43:56.063822 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.063804 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.075181 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.075155 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8px\" (UniqueName: \"kubernetes.io/projected/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-kube-api-access-7h8px\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.567058 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.567019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.569314 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.569295 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nkhnk\" (UID: \"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.831796 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.831713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" Jul 27 23:43:56.967669 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:56.967644 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk"] Jul 27 23:43:56.970252 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:43:56.970218 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc9a7bbf_d2d1_44f3_8c0f_b345081ac53b.slice/crio-48975ecb7a62b541078ce700e02044c925b7b00d3bb757fd0a364c4e62a0a371 WatchSource:0}: Error finding container 48975ecb7a62b541078ce700e02044c925b7b00d3bb757fd0a364c4e62a0a371: Status 404 returned error can't find the container with id 48975ecb7a62b541078ce700e02044c925b7b00d3bb757fd0a364c4e62a0a371 Jul 27 23:43:57.165124 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:43:57.165086 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" event={"ID":"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b","Type":"ContainerStarted","Data":"48975ecb7a62b541078ce700e02044c925b7b00d3bb757fd0a364c4e62a0a371"} Jul 27 23:44:05.911486 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:05.911449 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px"] Jul 27 23:44:05.921627 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:05.921566 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:05.924231 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:05.924205 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q94k8\"" Jul 27 23:44:05.924672 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:05.924651 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px"] Jul 27 23:44:06.059606 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.059570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhllh\" (UniqueName: \"kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.059767 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.059641 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.160682 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.160648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.160858 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.160795 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhllh\" (UniqueName: \"kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.161119 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.161070 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.177584 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.177509 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhllh\" (UniqueName: \"kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh\") pod \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.193555 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.193529 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px"] Jul 27 23:44:06.193850 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.193828 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:06.200794 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.200769 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px"] Jul 27 23:44:06.215395 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.215357 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:06.219123 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.219099 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.230202 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.230177 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:06.362646 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.362615 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.362825 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.362665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4cll\" (UniqueName: \"kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.463451 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.463358 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.463451 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.463417 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4cll\" (UniqueName: \"kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.463833 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.463812 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.478294 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.478264 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4cll\" (UniqueName: \"kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll\") pod \"kuadrant-operator-controller-manager-55467d9f7-qpxcg\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:06.533201 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:06.533169 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:18.499575 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:18.499543 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:18.502040 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:44:18.502013 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87205c2_0333_4765_ac14_27148d3403ee.slice/crio-895eea00ab36942f202fd168f79ab9ce98b0c99d54e9e8d1fb918d206d35b894 WatchSource:0}: Error finding container 895eea00ab36942f202fd168f79ab9ce98b0c99d54e9e8d1fb918d206d35b894: Status 404 returned error can't find the container with id 895eea00ab36942f202fd168f79ab9ce98b0c99d54e9e8d1fb918d206d35b894 Jul 27 23:44:19.282591 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:19.282548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" event={"ID":"cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b","Type":"ContainerStarted","Data":"7ee1c6cb3303ab24fd75e38b099e5c420e304e7549b51cdf84d50b520c992fce"} Jul 27 23:44:19.283826 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:19.283801 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" event={"ID":"d87205c2-0333-4765-ac14-27148d3403ee","Type":"ContainerStarted","Data":"895eea00ab36942f202fd168f79ab9ce98b0c99d54e9e8d1fb918d206d35b894"} Jul 27 23:44:19.296659 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:19.296535 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nkhnk" podStartSLOduration=2.8262391989999998 podStartE2EDuration="24.2965004s" totalImagesPullingTime="21.470261201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:44:19.29619548 +0000 UTC m=+434.656732153" watchObservedRunningTime="2026-07-27 23:44:19.2965004 +0000 UTC m=+434.657037079" Jul 27 23:44:21.144125 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:21.144072 2574 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-vt7px_kuadrant-system_d4487247-a725-4e4d-815d-371683084dda_0(d224b9040cdd7244cd20bcaa32c080fe851179fbb0b7ae829ae7fade061049d3): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-vt7px to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 27 23:44:21.144433 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:21.144151 2574 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-vt7px_kuadrant-system_d4487247-a725-4e4d-815d-371683084dda_0(d224b9040cdd7244cd20bcaa32c080fe851179fbb0b7ae829ae7fade061049d3): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-vt7px to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:21.303485 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.303445 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:21.305873 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:21.305830 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="d4487247-a725-4e4d-815d-371683084dda" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:21.330955 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.330932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:21.333096 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:21.333067 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="d4487247-a725-4e4d-815d-371683084dda" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:21.511502 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.511422 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume\") pod \"d4487247-a725-4e4d-815d-371683084dda\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " Jul 27 23:44:21.511685 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.511545 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhllh\" (UniqueName: \"kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh\") pod \"d4487247-a725-4e4d-815d-371683084dda\" (UID: \"d4487247-a725-4e4d-815d-371683084dda\") " Jul 27 23:44:21.511753 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.511722 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d4487247-a725-4e4d-815d-371683084dda" (UID: "d4487247-a725-4e4d-815d-371683084dda"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:44:21.513887 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.513857 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh" (OuterVolumeSpecName: "kube-api-access-lhllh") pod "d4487247-a725-4e4d-815d-371683084dda" (UID: "d4487247-a725-4e4d-815d-371683084dda"). InnerVolumeSpecName "kube-api-access-lhllh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:44:21.612613 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.612560 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhllh\" (UniqueName: \"kubernetes.io/projected/d4487247-a725-4e4d-815d-371683084dda-kube-api-access-lhllh\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:44:21.612613 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:21.612612 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4487247-a725-4e4d-815d-371683084dda-extensions-socket-volume\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:44:22.309671 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:22.309616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:22.311905 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:22.311868 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="d4487247-a725-4e4d-815d-371683084dda" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:22.319241 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:22.319210 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-vt7px\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="d4487247-a725-4e4d-815d-371683084dda" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vt7px" Jul 27 23:44:23.174773 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:23.174736 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4487247-a725-4e4d-815d-371683084dda" path="/var/lib/kubelet/pods/d4487247-a725-4e4d-815d-371683084dda/volumes" Jul 27 23:44:24.321696 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:24.321662 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" event={"ID":"d87205c2-0333-4765-ac14-27148d3403ee","Type":"ContainerStarted","Data":"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7"} Jul 27 23:44:24.322150 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:24.321794 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:24.340097 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:24.340051 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" podStartSLOduration=12.667556463 podStartE2EDuration="18.340036009s" totalImagesPullingTime="5.672479546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:44:24.339137507 +0000 UTC m=+439.699674197" watchObservedRunningTime="2026-07-27 23:44:24.340036009 +0000 UTC m=+439.700572686" Jul 27 23:44:35.329026 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:35.328950 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:48.492048 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.492016 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:48.492433 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.492213 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" podUID="d87205c2-0333-4765-ac14-27148d3403ee" containerName="manager" containerID="cri-o://90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7" gracePeriod=10 Jul 27 23:44:48.730368 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.730344 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:48.847460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.847372 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume\") pod \"d87205c2-0333-4765-ac14-27148d3403ee\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " Jul 27 23:44:48.847460 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.847411 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4cll\" (UniqueName: \"kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll\") pod \"d87205c2-0333-4765-ac14-27148d3403ee\" (UID: \"d87205c2-0333-4765-ac14-27148d3403ee\") " Jul 27 23:44:48.847933 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.847911 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d87205c2-0333-4765-ac14-27148d3403ee" (UID: "d87205c2-0333-4765-ac14-27148d3403ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:44:48.849702 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.849685 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll" (OuterVolumeSpecName: "kube-api-access-m4cll") pod "d87205c2-0333-4765-ac14-27148d3403ee" (UID: "d87205c2-0333-4765-ac14-27148d3403ee"). InnerVolumeSpecName "kube-api-access-m4cll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:44:48.948316 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.948267 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87205c2-0333-4765-ac14-27148d3403ee-extensions-socket-volume\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:44:48.948316 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:48.948310 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4cll\" (UniqueName: \"kubernetes.io/projected/d87205c2-0333-4765-ac14-27148d3403ee-kube-api-access-m4cll\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:44:49.435257 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.435173 2574 generic.go:356] "Generic (PLEG): container finished" podID="d87205c2-0333-4765-ac14-27148d3403ee" containerID="90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7" exitCode=0 Jul 27 23:44:49.435257 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.435224 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" Jul 27 23:44:49.435437 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.435255 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" event={"ID":"d87205c2-0333-4765-ac14-27148d3403ee","Type":"ContainerDied","Data":"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7"} Jul 27 23:44:49.435437 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.435309 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg" event={"ID":"d87205c2-0333-4765-ac14-27148d3403ee","Type":"ContainerDied","Data":"895eea00ab36942f202fd168f79ab9ce98b0c99d54e9e8d1fb918d206d35b894"} Jul 27 23:44:49.435437 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.435331 2574 scope.go:117] "RemoveContainer" containerID="90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7" Jul 27 23:44:49.444062 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.444037 2574 scope.go:117] "RemoveContainer" containerID="90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7" Jul 27 23:44:49.444348 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:44:49.444325 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7\": container with ID starting with 90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7 not found: ID does not exist" containerID="90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7" Jul 27 23:44:49.444429 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.444360 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7"} err="failed to get container status \"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7\": rpc error: code = NotFound desc = could not find container \"90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7\": container with ID starting with 90637753b49d2be1eebd09046fe44f024ca632c9644453a2efcd26ac47523ad7 not found: ID does not exist" Jul 27 23:44:49.455324 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.455302 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:49.462251 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:49.462230 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qpxcg"] Jul 27 23:44:51.172818 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:51.172785 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d87205c2-0333-4765-ac14-27148d3403ee" path="/var/lib/kubelet/pods/d87205c2-0333-4765-ac14-27148d3403ee/volumes" Jul 27 23:44:55.727475 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.727435 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:44:55.727866 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.727804 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d87205c2-0333-4765-ac14-27148d3403ee" containerName="manager" Jul 27 23:44:55.727866 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.727816 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d87205c2-0333-4765-ac14-27148d3403ee" containerName="manager" Jul 27 23:44:55.727943 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.727879 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d87205c2-0333-4765-ac14-27148d3403ee" containerName="manager" Jul 27 23:44:55.752686 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.752659 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:44:55.752842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.752697 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:44:55.752842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.752789 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:55.755037 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.755015 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 27 23:44:55.910920 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.910887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shm8f\" (UniqueName: \"kubernetes.io/projected/e5333281-b4b7-462f-aec7-e5a711662a18-kube-api-access-shm8f\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:55.911075 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:55.910936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e5333281-b4b7-462f-aec7-e5a711662a18-config-file\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.011768 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.011675 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shm8f\" (UniqueName: \"kubernetes.io/projected/e5333281-b4b7-462f-aec7-e5a711662a18-kube-api-access-shm8f\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.011768 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.011740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e5333281-b4b7-462f-aec7-e5a711662a18-config-file\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.012355 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.012335 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e5333281-b4b7-462f-aec7-e5a711662a18-config-file\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.019225 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.019204 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shm8f\" (UniqueName: \"kubernetes.io/projected/e5333281-b4b7-462f-aec7-e5a711662a18-kube-api-access-shm8f\") pod \"limitador-limitador-54d6d988d5-wp5vp\" (UID: \"e5333281-b4b7-462f-aec7-e5a711662a18\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.062715 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.062690 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:56.192504 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.192480 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:44:56.195261 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:44:56.195235 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5333281_b4b7_462f_aec7_e5a711662a18.slice/crio-c73f9df47647fc60bc0781fb10a7655a2f161f9d3f13a7021cea6413073d122d WatchSource:0}: Error finding container c73f9df47647fc60bc0781fb10a7655a2f161f9d3f13a7021cea6413073d122d: Status 404 returned error can't find the container with id c73f9df47647fc60bc0781fb10a7655a2f161f9d3f13a7021cea6413073d122d Jul 27 23:44:56.422224 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.422191 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:44:56.428500 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.428480 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:44:56.430919 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.430901 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b5mc4\"" Jul 27 23:44:56.432053 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.432027 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:44:56.468902 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.468871 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" event={"ID":"e5333281-b4b7-462f-aec7-e5a711662a18","Type":"ContainerStarted","Data":"c73f9df47647fc60bc0781fb10a7655a2f161f9d3f13a7021cea6413073d122d"} Jul 27 23:44:56.516370 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.516336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppsj\" (UniqueName: \"kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj\") pod \"authorino-75d556bd77-ptbnt\" (UID: \"5e5d8608-f098-4c4e-bd19-2a0783f92d6a\") " pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:44:56.617965 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.617929 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppsj\" (UniqueName: \"kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj\") pod \"authorino-75d556bd77-ptbnt\" (UID: \"5e5d8608-f098-4c4e-bd19-2a0783f92d6a\") " pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:44:56.627238 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.627200 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppsj\" (UniqueName: \"kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj\") pod \"authorino-75d556bd77-ptbnt\" (UID: \"5e5d8608-f098-4c4e-bd19-2a0783f92d6a\") " pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:44:56.739794 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.739542 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:44:56.931218 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:56.931187 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:44:56.933708 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:44:56.933674 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e5d8608_f098_4c4e_bd19_2a0783f92d6a.slice/crio-c432e8e72b6db28e5e1af798297c83515c0964c54c38a83cb3706060d4bc07c4 WatchSource:0}: Error finding container c432e8e72b6db28e5e1af798297c83515c0964c54c38a83cb3706060d4bc07c4: Status 404 returned error can't find the container with id c432e8e72b6db28e5e1af798297c83515c0964c54c38a83cb3706060d4bc07c4 Jul 27 23:44:57.476269 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:57.476235 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ptbnt" event={"ID":"5e5d8608-f098-4c4e-bd19-2a0783f92d6a","Type":"ContainerStarted","Data":"c432e8e72b6db28e5e1af798297c83515c0964c54c38a83cb3706060d4bc07c4"} Jul 27 23:44:59.491609 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:59.491562 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" event={"ID":"e5333281-b4b7-462f-aec7-e5a711662a18","Type":"ContainerStarted","Data":"78a67464aedf2a778956c93d19108f56d8bf13e90654cc82e1d926c468aabe23"} Jul 27 23:44:59.492098 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:59.491652 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:44:59.511362 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:44:59.511305 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" podStartSLOduration=1.794821381 podStartE2EDuration="4.511290203s" totalImagesPullingTime="2.716468822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:44:59.508744171 +0000 UTC m=+474.869280880" watchObservedRunningTime="2026-07-27 23:44:59.511290203 +0000 UTC m=+474.871826892" Jul 27 23:45:01.503681 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:01.503645 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ptbnt" event={"ID":"5e5d8608-f098-4c4e-bd19-2a0783f92d6a","Type":"ContainerStarted","Data":"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff"} Jul 27 23:45:01.519316 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:01.519270 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ptbnt" podStartSLOduration=1.6925856449999999 podStartE2EDuration="5.519255185s" totalImagesPullingTime="3.82666954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:45:01.517635345 +0000 UTC m=+476.878172025" watchObservedRunningTime="2026-07-27 23:45:01.519255185 +0000 UTC m=+476.879791913" Jul 27 23:45:10.504279 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:10.504243 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wp5vp" Jul 27 23:45:27.509105 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.509071 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:45:27.514123 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.514093 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.516350 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.516328 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 27 23:45:27.519480 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.519453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:45:27.593357 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.593320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhrd\" (UniqueName: \"kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.593508 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.593383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.693842 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.693807 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.694010 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.693897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhrd\" (UniqueName: \"kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.696186 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.696164 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.700837 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.700819 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhrd\" (UniqueName: \"kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd\") pod \"authorino-b5f5d876d-f7b8l\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.823638 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.823562 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:45:27.950971 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:27.950948 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:45:27.953067 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:45:27.953024 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1a0b261_f7a6_499a_864b_d84d08fcd6be.slice/crio-fa76c85e4b77c97a529dd2b16bd4aac90fd7d8308f9522fbd19d9040a36259c0 WatchSource:0}: Error finding container fa76c85e4b77c97a529dd2b16bd4aac90fd7d8308f9522fbd19d9040a36259c0: Status 404 returned error can't find the container with id fa76c85e4b77c97a529dd2b16bd4aac90fd7d8308f9522fbd19d9040a36259c0 Jul 27 23:45:28.629319 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.629242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" event={"ID":"c1a0b261-f7a6-499a-864b-d84d08fcd6be","Type":"ContainerStarted","Data":"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042"} Jul 27 23:45:28.629319 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.629282 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" event={"ID":"c1a0b261-f7a6-499a-864b-d84d08fcd6be","Type":"ContainerStarted","Data":"fa76c85e4b77c97a529dd2b16bd4aac90fd7d8308f9522fbd19d9040a36259c0"} Jul 27 23:45:28.645120 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.645063 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" podStartSLOduration=1.3214564979999999 podStartE2EDuration="1.64504714s" totalImagesPullingTime="323.590642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:45:28.643944224 +0000 UTC m=+504.004480902" watchObservedRunningTime="2026-07-27 23:45:28.64504714 +0000 UTC m=+504.005583818" Jul 27 23:45:28.667115 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.667080 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:45:28.667345 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.667316 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ptbnt" podUID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" containerName="authorino" containerID="cri-o://f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff" gracePeriod=30 Jul 27 23:45:28.918787 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:28.918762 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:45:29.004269 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.004240 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cppsj\" (UniqueName: \"kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj\") pod \"5e5d8608-f098-4c4e-bd19-2a0783f92d6a\" (UID: \"5e5d8608-f098-4c4e-bd19-2a0783f92d6a\") " Jul 27 23:45:29.006265 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.006239 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj" (OuterVolumeSpecName: "kube-api-access-cppsj") pod "5e5d8608-f098-4c4e-bd19-2a0783f92d6a" (UID: "5e5d8608-f098-4c4e-bd19-2a0783f92d6a"). InnerVolumeSpecName "kube-api-access-cppsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:45:29.105104 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.105065 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cppsj\" (UniqueName: \"kubernetes.io/projected/5e5d8608-f098-4c4e-bd19-2a0783f92d6a-kube-api-access-cppsj\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:45:29.634940 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.634905 2574 generic.go:356] "Generic (PLEG): container finished" podID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" containerID="f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff" exitCode=0 Jul 27 23:45:29.635313 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.634945 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ptbnt" Jul 27 23:45:29.635313 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.634987 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ptbnt" event={"ID":"5e5d8608-f098-4c4e-bd19-2a0783f92d6a","Type":"ContainerDied","Data":"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff"} Jul 27 23:45:29.635313 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.635026 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ptbnt" event={"ID":"5e5d8608-f098-4c4e-bd19-2a0783f92d6a","Type":"ContainerDied","Data":"c432e8e72b6db28e5e1af798297c83515c0964c54c38a83cb3706060d4bc07c4"} Jul 27 23:45:29.635313 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.635050 2574 scope.go:117] "RemoveContainer" containerID="f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff" Jul 27 23:45:29.643371 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.643347 2574 scope.go:117] "RemoveContainer" containerID="f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff" Jul 27 23:45:29.643639 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:45:29.643620 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff\": container with ID starting with f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff not found: ID does not exist" containerID="f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff" Jul 27 23:45:29.643711 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.643650 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff"} err="failed to get container status \"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff\": rpc error: code = NotFound desc = could not find container \"f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff\": container with ID starting with f7006bf378f51536cfe5f5b5fb05c20ed2f5c5753b74b2c6b5b658314ac09dff not found: ID does not exist" Jul 27 23:45:29.656488 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.656460 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:45:29.660376 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:29.660358 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ptbnt"] Jul 27 23:45:31.172770 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:45:31.172735 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" path="/var/lib/kubelet/pods/5e5d8608-f098-4c4e-bd19-2a0783f92d6a/volumes" Jul 27 23:46:10.382398 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:46:10.382361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:46:26.335838 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:46:26.335793 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:46:32.137024 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:46:32.136985 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:46:40.554426 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:46:40.554389 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:46:55.538230 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:46:55.538189 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:47:05.040917 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:47:05.040884 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:47:05.045093 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:47:05.045067 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:47:05.049843 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:47:05.049822 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:47:14.332566 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:47:14.332530 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:48:33.191879 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.191850 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c956cb97-bskpx"] Jul 27 23:48:33.192328 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.192221 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" containerName="authorino" Jul 27 23:48:33.192328 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.192233 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" containerName="authorino" Jul 27 23:48:33.192328 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.192312 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e5d8608-f098-4c4e-bd19-2a0783f92d6a" containerName="authorino" Jul 27 23:48:33.194969 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.194954 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.205144 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.205120 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c956cb97-bskpx"] Jul 27 23:48:33.253067 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.252783 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2594fedd-1cfd-4885-bcda-31eb5e900d33-tls-cert\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.253067 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.252977 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx5vr\" (UniqueName: \"kubernetes.io/projected/2594fedd-1cfd-4885-bcda-31eb5e900d33-kube-api-access-dx5vr\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.354600 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.354562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2594fedd-1cfd-4885-bcda-31eb5e900d33-tls-cert\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.354765 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.354619 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5vr\" (UniqueName: \"kubernetes.io/projected/2594fedd-1cfd-4885-bcda-31eb5e900d33-kube-api-access-dx5vr\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.356978 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.356960 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2594fedd-1cfd-4885-bcda-31eb5e900d33-tls-cert\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.362355 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.362334 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx5vr\" (UniqueName: \"kubernetes.io/projected/2594fedd-1cfd-4885-bcda-31eb5e900d33-kube-api-access-dx5vr\") pod \"authorino-59c956cb97-bskpx\" (UID: \"2594fedd-1cfd-4885-bcda-31eb5e900d33\") " pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.503396 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.503329 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c956cb97-bskpx" Jul 27 23:48:33.637547 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.634732 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c956cb97-bskpx"] Jul 27 23:48:33.641227 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:48:33.641201 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2594fedd_1cfd_4885_bcda_31eb5e900d33.slice/crio-473d560d7954e3107254395ea3d2acb1ee2d562eee3c5e9abcffd91a38fe08b9 WatchSource:0}: Error finding container 473d560d7954e3107254395ea3d2acb1ee2d562eee3c5e9abcffd91a38fe08b9: Status 404 returned error can't find the container with id 473d560d7954e3107254395ea3d2acb1ee2d562eee3c5e9abcffd91a38fe08b9 Jul 27 23:48:33.642444 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:33.642428 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:48:34.485499 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.485464 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c956cb97-bskpx" event={"ID":"2594fedd-1cfd-4885-bcda-31eb5e900d33","Type":"ContainerStarted","Data":"bf95d8b86dc7fbaffa62003be4aca2986e3b4543b544aea2b33da84b0a945305"} Jul 27 23:48:34.485499 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.485507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c956cb97-bskpx" event={"ID":"2594fedd-1cfd-4885-bcda-31eb5e900d33","Type":"ContainerStarted","Data":"473d560d7954e3107254395ea3d2acb1ee2d562eee3c5e9abcffd91a38fe08b9"} Jul 27 23:48:34.500412 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.500362 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c956cb97-bskpx" podStartSLOduration=0.837602197 podStartE2EDuration="1.500343716s" totalImagesPullingTime="662.741519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:48:34.498239941 +0000 UTC m=+689.858776639" watchObservedRunningTime="2026-07-27 23:48:34.500343716 +0000 UTC m=+689.860880395" Jul 27 23:48:34.529307 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.529271 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:48:34.529753 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.529499 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" podUID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" containerName="authorino" containerID="cri-o://d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042" gracePeriod=30 Jul 27 23:48:34.793961 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.793937 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:48:34.868222 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.868142 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfhrd\" (UniqueName: \"kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd\") pod \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " Jul 27 23:48:34.868367 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.868228 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert\") pod \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\" (UID: \"c1a0b261-f7a6-499a-864b-d84d08fcd6be\") " Jul 27 23:48:34.871615 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.871579 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd" (OuterVolumeSpecName: "kube-api-access-cfhrd") pod "c1a0b261-f7a6-499a-864b-d84d08fcd6be" (UID: "c1a0b261-f7a6-499a-864b-d84d08fcd6be"). InnerVolumeSpecName "kube-api-access-cfhrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:48:34.879190 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.879157 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c1a0b261-f7a6-499a-864b-d84d08fcd6be" (UID: "c1a0b261-f7a6-499a-864b-d84d08fcd6be"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:48:34.969331 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.969299 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1a0b261-f7a6-499a-864b-d84d08fcd6be-tls-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:48:34.969331 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:34.969329 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfhrd\" (UniqueName: \"kubernetes.io/projected/c1a0b261-f7a6-499a-864b-d84d08fcd6be-kube-api-access-cfhrd\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 27 23:48:35.491849 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.491751 2574 generic.go:356] "Generic (PLEG): container finished" podID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" containerID="d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042" exitCode=0 Jul 27 23:48:35.491849 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.491792 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" Jul 27 23:48:35.491849 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.491835 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" event={"ID":"c1a0b261-f7a6-499a-864b-d84d08fcd6be","Type":"ContainerDied","Data":"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042"} Jul 27 23:48:35.492369 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.491875 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f5d876d-f7b8l" event={"ID":"c1a0b261-f7a6-499a-864b-d84d08fcd6be","Type":"ContainerDied","Data":"fa76c85e4b77c97a529dd2b16bd4aac90fd7d8308f9522fbd19d9040a36259c0"} Jul 27 23:48:35.492369 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.491895 2574 scope.go:117] "RemoveContainer" containerID="d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042" Jul 27 23:48:35.500872 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.500851 2574 scope.go:117] "RemoveContainer" containerID="d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042" Jul 27 23:48:35.501217 ip-10-0-128-69 kubenswrapper[2574]: E0727 23:48:35.501191 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042\": container with ID starting with d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042 not found: ID does not exist" containerID="d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042" Jul 27 23:48:35.501272 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.501228 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042"} err="failed to get container status \"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042\": rpc error: code = NotFound desc = could not find container \"d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042\": container with ID starting with d7c8a9abc5c2f49dac738d941c293870bfb2731fcd360bb9ca9e6c61a7f99042 not found: ID does not exist" Jul 27 23:48:35.510303 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.510279 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:48:35.513625 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:35.513599 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b5f5d876d-f7b8l"] Jul 27 23:48:37.173955 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:48:37.173923 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" path="/var/lib/kubelet/pods/c1a0b261-f7a6-499a-864b-d84d08fcd6be/volumes" Jul 27 23:49:05.337162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:49:05.337082 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:49:15.033802 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:49:15.033762 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:49:25.339725 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:49:25.339689 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:51:07.035462 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:51:07.035426 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:51:15.933533 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:51:15.933489 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:51:45.436123 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:51:45.436077 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:51:54.635848 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:51:54.635814 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:03.050431 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:03.050393 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:05.077620 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:05.077591 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:52:05.078625 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:05.078603 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:52:12.734833 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:12.734800 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:23.039769 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:23.039733 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:32.236708 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:32.236672 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:43.837387 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:43.837347 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:52:53.961702 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:52:53.961668 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:53:16.134490 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:53:16.134455 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:53:27.839167 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:53:27.839099 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:53:51.237449 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:53:51.237414 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:01.439355 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:01.439307 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:11.339960 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:11.339929 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:28.140746 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:28.140712 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:37.643094 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:37.643057 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:47.151411 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:47.151371 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:54:57.229896 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:54:57.229817 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:55:06.736937 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:55:06.736901 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:56:33.870476 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.870384 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw"] Jul 27 23:56:33.870976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.870886 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" containerName="authorino" Jul 27 23:56:33.870976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.870902 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" containerName="authorino" Jul 27 23:56:33.870976 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.870972 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1a0b261-f7a6-499a-864b-d84d08fcd6be" containerName="authorino" Jul 27 23:56:33.874120 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.874102 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:33.876799 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.876777 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q94k8\"" Jul 27 23:56:33.892607 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.892576 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw"] Jul 27 23:56:33.920245 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.920217 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk889\" (UniqueName: \"kubernetes.io/projected/925ffbce-aa7e-4905-9b5e-639429fcbbc7-kube-api-access-lk889\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:33.920365 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:33.920273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925ffbce-aa7e-4905-9b5e-639429fcbbc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.021033 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.020994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk889\" (UniqueName: \"kubernetes.io/projected/925ffbce-aa7e-4905-9b5e-639429fcbbc7-kube-api-access-lk889\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.021184 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.021047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925ffbce-aa7e-4905-9b5e-639429fcbbc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.021405 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.021384 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925ffbce-aa7e-4905-9b5e-639429fcbbc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.029819 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.029796 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk889\" (UniqueName: \"kubernetes.io/projected/925ffbce-aa7e-4905-9b5e-639429fcbbc7-kube-api-access-lk889\") pod \"kuadrant-operator-controller-manager-86878c769f-jlmdw\" (UID: \"925ffbce-aa7e-4905-9b5e-639429fcbbc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.184368 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.184339 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.326603 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.326566 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw"] Jul 27 23:56:34.332674 ip-10-0-128-69 kubenswrapper[2574]: W0727 23:56:34.332648 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925ffbce_aa7e_4905_9b5e_639429fcbbc7.slice/crio-f041e331da6059adb1cdd4167ba96ac30d044e29efe444f99c50c0ba26683aa8 WatchSource:0}: Error finding container f041e331da6059adb1cdd4167ba96ac30d044e29efe444f99c50c0ba26683aa8: Status 404 returned error can't find the container with id f041e331da6059adb1cdd4167ba96ac30d044e29efe444f99c50c0ba26683aa8 Jul 27 23:56:34.725083 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.725043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" event={"ID":"925ffbce-aa7e-4905-9b5e-639429fcbbc7","Type":"ContainerStarted","Data":"6394734cc85212e4aa425b6f9577a2e3f68cc1543101fd5d0b335ec8992a9209"} Jul 27 23:56:34.725255 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.725111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" event={"ID":"925ffbce-aa7e-4905-9b5e-639429fcbbc7","Type":"ContainerStarted","Data":"f041e331da6059adb1cdd4167ba96ac30d044e29efe444f99c50c0ba26683aa8"} Jul 27 23:56:34.725255 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.725147 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:34.750734 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:34.750681 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" podStartSLOduration=1.750666325 podStartE2EDuration="1.750666325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:56:34.74808068 +0000 UTC m=+1170.108617359" watchObservedRunningTime="2026-07-27 23:56:34.750666325 +0000 UTC m=+1170.111202981" Jul 27 23:56:45.731888 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:45.731858 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jlmdw" Jul 27 23:56:55.777944 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:56:55.777906 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:57:05.109699 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:05.109672 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:57:05.110262 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:05.110229 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 27 23:57:07.068095 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:07.068060 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:57:22.582741 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:22.582699 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:57:31.574893 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:31.574860 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:57:43.976090 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:43.976057 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:57:51.874239 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:57:51.874201 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:02.573115 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:02.573041 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:14.581132 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:14.581097 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:22.681037 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:22.680991 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:34.376086 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:34.376052 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:45.283143 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:45.283105 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:58:53.293162 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:58:53.293126 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:59:11.182434 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:59:11.182401 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:59:20.782572 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:59:20.782534 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:59:34.080684 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:59:34.080651 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 27 23:59:53.183741 ip-10-0-128-69 kubenswrapper[2574]: I0727 23:59:53.183705 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:00:24.883803 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:00:24.883768 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:00:39.469279 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:00:39.469243 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:00:46.177479 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:00:46.177431 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:02.287566 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:02.287452 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:12.975118 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:12.975082 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:22.077357 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:22.077322 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:32.283565 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:32.283502 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:41.377284 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:41.377246 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:01:50.778620 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:01:50.778585 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:05.075772 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:05.075740 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:05.144907 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:05.144868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:02:05.145928 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:05.145905 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:02:14.181015 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:14.180981 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:23.772071 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:23.772032 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:33.175425 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:33.175392 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:45.870548 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:45.870498 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:02:54.572335 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:02:54.572301 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:06.475727 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:06.475695 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:19.081828 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:19.081791 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:28.180906 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:28.180870 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:37.676985 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:37.676949 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:50.877645 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:50.877611 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:03:59.381005 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:03:59.380929 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:04:10.976154 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:04:10.976120 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:04:20.676582 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:04:20.676545 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:04:52.279262 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:04:52.279227 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:05:05.375073 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:05:05.375039 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:05:35.975266 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:05:35.975179 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:05:49.082272 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:05:49.082234 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:06:21.178377 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:06:21.178343 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:06:34.077059 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:06:34.077022 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:07:04.582121 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:07:04.582085 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:07:05.179602 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:07:05.179574 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:07:05.180958 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:07:05.180940 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:08:32.074298 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:08:32.074215 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:08:41.277666 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:08:41.277624 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:08:53.174267 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:08:53.174226 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:09:16.076313 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:09:16.076277 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:09:53.580692 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:09:53.580654 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:10:02.684012 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:10:02.683975 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:10:21.676076 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:10:21.676034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:10:40.517584 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:10:40.517549 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:10:48.975916 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:10:48.975878 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:11:46.686069 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:11:46.686036 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:11:58.579015 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:11:58.578932 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:12:05.206112 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:12:05.206081 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:12:05.208479 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:12:05.208461 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:12:47.982559 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:12:47.982505 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:12:56.583587 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:12:56.583546 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:13:10.076339 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:13:10.076292 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:13:38.485632 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:13:38.485594 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:13:55.673285 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:13:55.673254 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:14:20.179061 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:14:20.179024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:14:32.679489 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:14:32.679401 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:14:51.782256 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:14:51.782221 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:15:11.775454 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:15:11.775419 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:15:26.374671 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:15:26.374634 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:15:43.179074 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:15:43.179038 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:16:10.481715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:16:10.481678 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:16:27.388051 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:16:27.388017 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:16:36.077510 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:16:36.077469 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:16:55.091796 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:16:55.091759 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:17:05.236843 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:17:05.236814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:17:05.238684 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:17:05.238650 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:17:26.676924 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:17:26.676846 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:17:56.576683 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:17:56.576646 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:18:17.580310 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:18:17.580271 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:18:35.179640 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:18:35.179601 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:18:59.179568 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:18:59.179480 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:19:15.671754 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:19:15.671714 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:19:38.380199 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:19:38.380168 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:19:55.874081 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:19:55.874031 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:20:23.769427 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:20:23.769388 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:20:41.868438 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:20:41.868407 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:21:04.281928 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:21:04.281896 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:21:21.077120 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:21:21.077078 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:21:41.671864 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:21:41.671823 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:21:58.169328 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:21:58.169290 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:22:05.265865 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:22:05.265837 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:22:05.268462 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:22:05.268440 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:22:11.175271 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:22:11.175235 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:22:22.983057 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:22:22.983024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:22:56.683611 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:22:56.683576 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:23:13.184402 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:23:13.184366 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:23:28.181906 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:23:28.181827 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:23:46.179014 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:23:46.178973 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:23:55.685482 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:23:55.685441 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:24:36.876203 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:24:36.876167 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:25:09.379677 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:09.379642 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:25:23.878420 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:23.878383 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:25:38.472151 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:38.472115 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:25:44.635721 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.635684 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999352960304"] Jul 28 00:25:44.639018 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.638998 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:44.641301 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.641275 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kxbjz\"" Jul 28 00:25:44.644936 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.644903 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999352960304"] Jul 28 00:25:44.775651 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.775613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbwnx\" (UniqueName: \"kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx\") pod \"test-curl-7406-139999352960304\" (UID: \"cb9b74e1-f434-4d32-a183-9efa8fa547ac\") " pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:44.876642 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.876601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbwnx\" (UniqueName: \"kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx\") pod \"test-curl-7406-139999352960304\" (UID: \"cb9b74e1-f434-4d32-a183-9efa8fa547ac\") " pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:44.884301 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.884269 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbwnx\" (UniqueName: \"kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx\") pod \"test-curl-7406-139999352960304\" (UID: \"cb9b74e1-f434-4d32-a183-9efa8fa547ac\") " pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:44.949772 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:44.949691 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:45.082187 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:45.082162 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999352960304"] Jul 28 00:25:45.084891 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:45.084861 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb9b74e1_f434_4d32_a183_9efa8fa547ac.slice/crio-ad56436e31140021ac89c66419011519a1a2ba1ce49a776984866bd09753c587 WatchSource:0}: Error finding container ad56436e31140021ac89c66419011519a1a2ba1ce49a776984866bd09753c587: Status 404 returned error can't find the container with id ad56436e31140021ac89c66419011519a1a2ba1ce49a776984866bd09753c587 Jul 28 00:25:45.086161 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:45.086136 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 00:25:45.940432 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:45.940393 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb9b74e1-f434-4d32-a183-9efa8fa547ac" containerID="1b73d13cbf9e59f1e951bb586ad569e5ac634defa63c6fddfdcbac683ac7becb" exitCode=0 Jul 28 00:25:45.940845 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:45.940471 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999352960304" event={"ID":"cb9b74e1-f434-4d32-a183-9efa8fa547ac","Type":"ContainerDied","Data":"1b73d13cbf9e59f1e951bb586ad569e5ac634defa63c6fddfdcbac683ac7becb"} Jul 28 00:25:45.940845 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:45.940531 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999352960304" event={"ID":"cb9b74e1-f434-4d32-a183-9efa8fa547ac","Type":"ContainerStarted","Data":"ad56436e31140021ac89c66419011519a1a2ba1ce49a776984866bd09753c587"} Jul 28 00:25:47.083458 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.083428 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:47.117814 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.117787 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999352960304_cb9b74e1-f434-4d32-a183-9efa8fa547ac/test-curl-7406-139999352960304/0.log" Jul 28 00:25:47.140974 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.140937 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999352960304"] Jul 28 00:25:47.145253 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.145212 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999352960304"] Jul 28 00:25:47.200350 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.200253 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbwnx\" (UniqueName: \"kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx\") pod \"cb9b74e1-f434-4d32-a183-9efa8fa547ac\" (UID: \"cb9b74e1-f434-4d32-a183-9efa8fa547ac\") " Jul 28 00:25:47.202553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.202523 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx" (OuterVolumeSpecName: "kube-api-access-vbwnx") pod "cb9b74e1-f434-4d32-a183-9efa8fa547ac" (UID: "cb9b74e1-f434-4d32-a183-9efa8fa547ac"). InnerVolumeSpecName "kube-api-access-vbwnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:25:47.264328 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.264291 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999349334320"] Jul 28 00:25:47.264741 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.264723 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb9b74e1-f434-4d32-a183-9efa8fa547ac" containerName="test-curl-7406-139999352960304" Jul 28 00:25:47.264823 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.264743 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb9b74e1-f434-4d32-a183-9efa8fa547ac" containerName="test-curl-7406-139999352960304" Jul 28 00:25:47.264887 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.264873 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb9b74e1-f434-4d32-a183-9efa8fa547ac" containerName="test-curl-7406-139999352960304" Jul 28 00:25:47.267894 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.267873 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:47.271568 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.271542 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349334320"] Jul 28 00:25:47.301411 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.301370 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbwnx\" (UniqueName: \"kubernetes.io/projected/cb9b74e1-f434-4d32-a183-9efa8fa547ac-kube-api-access-vbwnx\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:25:47.402097 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.402050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428pr\" (UniqueName: \"kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr\") pod \"test-curl-7406-139999349334320\" (UID: \"7801b52d-854e-413d-b41b-363c6d04a3af\") " pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:47.503161 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.503068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428pr\" (UniqueName: \"kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr\") pod \"test-curl-7406-139999349334320\" (UID: \"7801b52d-854e-413d-b41b-363c6d04a3af\") " pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:47.514026 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.513989 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-428pr\" (UniqueName: \"kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr\") pod \"test-curl-7406-139999349334320\" (UID: \"7801b52d-854e-413d-b41b-363c6d04a3af\") " pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:47.579629 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.579595 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:47.710175 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.710138 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349334320"] Jul 28 00:25:47.712352 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:47.712325 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7801b52d_854e_413d_b41b_363c6d04a3af.slice/crio-51ad65fbe8eb4eae502c5ec45e27e2c96ae08eb7ca73b8a2e7d68e0488bd37a5 WatchSource:0}: Error finding container 51ad65fbe8eb4eae502c5ec45e27e2c96ae08eb7ca73b8a2e7d68e0488bd37a5: Status 404 returned error can't find the container with id 51ad65fbe8eb4eae502c5ec45e27e2c96ae08eb7ca73b8a2e7d68e0488bd37a5 Jul 28 00:25:47.951735 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.951708 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999352960304" Jul 28 00:25:47.951901 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.951715 2574 scope.go:117] "RemoveContainer" containerID="1b73d13cbf9e59f1e951bb586ad569e5ac634defa63c6fddfdcbac683ac7becb" Jul 28 00:25:47.953021 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:47.952943 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349334320" event={"ID":"7801b52d-854e-413d-b41b-363c6d04a3af","Type":"ContainerStarted","Data":"51ad65fbe8eb4eae502c5ec45e27e2c96ae08eb7ca73b8a2e7d68e0488bd37a5"} Jul 28 00:25:48.959645 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:48.959616 2574 generic.go:356] "Generic (PLEG): container finished" podID="7801b52d-854e-413d-b41b-363c6d04a3af" containerID="05fe5f17bfc8dc3e0c70eadacb78ac6ba45ad1a6fb309027674d004152ed49ba" exitCode=0 Jul 28 00:25:48.960030 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:48.959712 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349334320" event={"ID":"7801b52d-854e-413d-b41b-363c6d04a3af","Type":"ContainerDied","Data":"05fe5f17bfc8dc3e0c70eadacb78ac6ba45ad1a6fb309027674d004152ed49ba"} Jul 28 00:25:49.174245 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:49.174204 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb9b74e1-f434-4d32-a183-9efa8fa547ac" path="/var/lib/kubelet/pods/cb9b74e1-f434-4d32-a183-9efa8fa547ac/volumes" Jul 28 00:25:50.094332 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.094308 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:50.115465 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.115440 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999349334320_7801b52d-854e-413d-b41b-363c6d04a3af/test-curl-7406-139999349334320/0.log" Jul 28 00:25:50.140671 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.140631 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999349334320"] Jul 28 00:25:50.143711 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.143680 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999349334320"] Jul 28 00:25:50.229859 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.229767 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-428pr\" (UniqueName: \"kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr\") pod \"7801b52d-854e-413d-b41b-363c6d04a3af\" (UID: \"7801b52d-854e-413d-b41b-363c6d04a3af\") " Jul 28 00:25:50.231871 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.231837 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr" (OuterVolumeSpecName: "kube-api-access-428pr") pod "7801b52d-854e-413d-b41b-363c6d04a3af" (UID: "7801b52d-854e-413d-b41b-363c6d04a3af"). InnerVolumeSpecName "kube-api-access-428pr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:25:50.302375 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.302337 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:50.302737 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.302723 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7801b52d-854e-413d-b41b-363c6d04a3af" containerName="test-curl-7406-139999349334320" Jul 28 00:25:50.302785 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.302738 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7801b52d-854e-413d-b41b-363c6d04a3af" containerName="test-curl-7406-139999349334320" Jul 28 00:25:50.302823 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.302801 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7801b52d-854e-413d-b41b-363c6d04a3af" containerName="test-curl-7406-139999349334320" Jul 28 00:25:50.305745 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.305730 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:50.309177 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.309147 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:50.330627 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.330595 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-428pr\" (UniqueName: \"kubernetes.io/projected/7801b52d-854e-413d-b41b-363c6d04a3af-kube-api-access-428pr\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:25:50.431576 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.431534 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prrxd\" (UniqueName: \"kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd\") pod \"test-curl-7406-139999349291024\" (UID: \"e4494cae-d1fb-4771-96db-0ac7056647a4\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:50.533192 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.533106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prrxd\" (UniqueName: \"kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd\") pod \"test-curl-7406-139999349291024\" (UID: \"e4494cae-d1fb-4771-96db-0ac7056647a4\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:50.541109 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.541075 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prrxd\" (UniqueName: \"kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd\") pod \"test-curl-7406-139999349291024\" (UID: \"e4494cae-d1fb-4771-96db-0ac7056647a4\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:50.615606 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.615569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:50.750764 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.750613 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:50.753337 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:50.753308 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4494cae_d1fb_4771_96db_0ac7056647a4.slice/crio-98adfbfcb72da17cf9ac112c8d527a085d0271b5cc421d0f1428c66f66565e62 WatchSource:0}: Error finding container 98adfbfcb72da17cf9ac112c8d527a085d0271b5cc421d0f1428c66f66565e62: Status 404 returned error can't find the container with id 98adfbfcb72da17cf9ac112c8d527a085d0271b5cc421d0f1428c66f66565e62 Jul 28 00:25:50.971581 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.971541 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349291024" event={"ID":"e4494cae-d1fb-4771-96db-0ac7056647a4","Type":"ContainerStarted","Data":"98adfbfcb72da17cf9ac112c8d527a085d0271b5cc421d0f1428c66f66565e62"} Jul 28 00:25:50.972795 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.972771 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51ad65fbe8eb4eae502c5ec45e27e2c96ae08eb7ca73b8a2e7d68e0488bd37a5" Jul 28 00:25:50.972907 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:50.972795 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:50.983404 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:25:50.983372 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7406-139999349334320\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="7801b52d-854e-413d-b41b-363c6d04a3af" pod="opendatahub/test-curl-7406-139999349334320" Jul 28 00:25:51.173773 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:51.173743 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7801b52d-854e-413d-b41b-363c6d04a3af" path="/var/lib/kubelet/pods/7801b52d-854e-413d-b41b-363c6d04a3af/volumes" Jul 28 00:25:51.978575 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:51.978539 2574 generic.go:356] "Generic (PLEG): container finished" podID="e4494cae-d1fb-4771-96db-0ac7056647a4" containerID="33fe87ccae38420dfa2e93969df45e33554d170d1b815d25f0f0195c1879ce91" exitCode=0 Jul 28 00:25:51.978721 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:51.978621 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349291024" event={"ID":"e4494cae-d1fb-4771-96db-0ac7056647a4","Type":"ContainerDied","Data":"33fe87ccae38420dfa2e93969df45e33554d170d1b815d25f0f0195c1879ce91"} Jul 28 00:25:53.118033 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.118003 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.137130 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.137102 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999349291024_e4494cae-d1fb-4771-96db-0ac7056647a4/test-curl-7406-139999349291024/0.log" Jul 28 00:25:53.159636 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.159599 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:53.163413 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.163385 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:53.257899 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.257800 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prrxd\" (UniqueName: \"kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd\") pod \"e4494cae-d1fb-4771-96db-0ac7056647a4\" (UID: \"e4494cae-d1fb-4771-96db-0ac7056647a4\") " Jul 28 00:25:53.259949 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.259925 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd" (OuterVolumeSpecName: "kube-api-access-prrxd") pod "e4494cae-d1fb-4771-96db-0ac7056647a4" (UID: "e4494cae-d1fb-4771-96db-0ac7056647a4"). InnerVolumeSpecName "kube-api-access-prrxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:25:53.290705 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.290670 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:53.291042 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.291029 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4494cae-d1fb-4771-96db-0ac7056647a4" containerName="test-curl-7406-139999349291024" Jul 28 00:25:53.291081 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.291044 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4494cae-d1fb-4771-96db-0ac7056647a4" containerName="test-curl-7406-139999349291024" Jul 28 00:25:53.291114 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.291110 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4494cae-d1fb-4771-96db-0ac7056647a4" containerName="test-curl-7406-139999349291024" Jul 28 00:25:53.294021 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.294005 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.299877 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.299850 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:53.358981 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.358942 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prrxd\" (UniqueName: \"kubernetes.io/projected/e4494cae-d1fb-4771-96db-0ac7056647a4-kube-api-access-prrxd\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:25:53.382261 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.382226 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:25:53.460078 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.460038 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzmkt\" (UniqueName: \"kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt\") pod \"test-curl-7406-139999349291024\" (UID: \"17acce5c-579e-4990-9003-96960aa39ed7\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.561008 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.560919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzmkt\" (UniqueName: \"kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt\") pod \"test-curl-7406-139999349291024\" (UID: \"17acce5c-579e-4990-9003-96960aa39ed7\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.569226 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.569207 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzmkt\" (UniqueName: \"kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt\") pod \"test-curl-7406-139999349291024\" (UID: \"17acce5c-579e-4990-9003-96960aa39ed7\") " pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.604431 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.604398 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.736345 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.736315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:53.738753 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:53.738728 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17acce5c_579e_4990_9003_96960aa39ed7.slice/crio-1b9a37a8a1a4f4f13a95b97af3fddb62b028474b09e30a3a02475f5f16ccf8bf WatchSource:0}: Error finding container 1b9a37a8a1a4f4f13a95b97af3fddb62b028474b09e30a3a02475f5f16ccf8bf: Status 404 returned error can't find the container with id 1b9a37a8a1a4f4f13a95b97af3fddb62b028474b09e30a3a02475f5f16ccf8bf Jul 28 00:25:53.989383 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.989351 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:53.989579 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.989358 2574 scope.go:117] "RemoveContainer" containerID="33fe87ccae38420dfa2e93969df45e33554d170d1b815d25f0f0195c1879ce91" Jul 28 00:25:53.990722 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:53.990628 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349291024" event={"ID":"17acce5c-579e-4990-9003-96960aa39ed7","Type":"ContainerStarted","Data":"1b9a37a8a1a4f4f13a95b97af3fddb62b028474b09e30a3a02475f5f16ccf8bf"} Jul 28 00:25:54.997392 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:54.997351 2574 generic.go:356] "Generic (PLEG): container finished" podID="17acce5c-579e-4990-9003-96960aa39ed7" containerID="61984096b522c4acdb4d24d8aa9d2412a48e7dbbf236db2e9f82502decffa2e5" exitCode=0 Jul 28 00:25:54.997819 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:54.997407 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349291024" event={"ID":"17acce5c-579e-4990-9003-96960aa39ed7","Type":"ContainerDied","Data":"61984096b522c4acdb4d24d8aa9d2412a48e7dbbf236db2e9f82502decffa2e5"} Jul 28 00:25:55.178581 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:55.178545 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4494cae-d1fb-4771-96db-0ac7056647a4" path="/var/lib/kubelet/pods/e4494cae-d1fb-4771-96db-0ac7056647a4/volumes" Jul 28 00:25:56.141640 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.141609 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:56.160473 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.160450 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999349291024_17acce5c-579e-4990-9003-96960aa39ed7/test-curl-7406-139999349291024/0.log" Jul 28 00:25:56.182534 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.182493 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:56.185966 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.185942 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999349291024"] Jul 28 00:25:56.286954 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.286865 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzmkt\" (UniqueName: \"kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt\") pod \"17acce5c-579e-4990-9003-96960aa39ed7\" (UID: \"17acce5c-579e-4990-9003-96960aa39ed7\") " Jul 28 00:25:56.288921 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.288897 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt" (OuterVolumeSpecName: "kube-api-access-kzmkt") pod "17acce5c-579e-4990-9003-96960aa39ed7" (UID: "17acce5c-579e-4990-9003-96960aa39ed7"). InnerVolumeSpecName "kube-api-access-kzmkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:25:56.304438 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.304409 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:56.304800 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.304785 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17acce5c-579e-4990-9003-96960aa39ed7" containerName="test-curl-7406-139999349291024" Jul 28 00:25:56.304838 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.304803 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="17acce5c-579e-4990-9003-96960aa39ed7" containerName="test-curl-7406-139999349291024" Jul 28 00:25:56.304882 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.304870 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="17acce5c-579e-4990-9003-96960aa39ed7" containerName="test-curl-7406-139999349291024" Jul 28 00:25:56.307760 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.307745 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:56.313365 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.313330 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:56.387891 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.387852 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzmkt\" (UniqueName: \"kubernetes.io/projected/17acce5c-579e-4990-9003-96960aa39ed7-kube-api-access-kzmkt\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:25:56.488335 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.488295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6g9z\" (UniqueName: \"kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z\") pod \"test-curl-7406-139999349292032\" (UID: \"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:56.589844 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.589750 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6g9z\" (UniqueName: \"kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z\") pod \"test-curl-7406-139999349292032\" (UID: \"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:56.597941 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.597916 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6g9z\" (UniqueName: \"kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z\") pod \"test-curl-7406-139999349292032\" (UID: \"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:56.617946 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.617893 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:56.750526 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:56.750489 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:56.752292 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:56.752257 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76d4b5c7_9c7b_4ade_b324_b12c1ea358d2.slice/crio-16916b0d92daf26e5ed6eb43f8926f96eb497b3deba66eca3e9329105a0793ac WatchSource:0}: Error finding container 16916b0d92daf26e5ed6eb43f8926f96eb497b3deba66eca3e9329105a0793ac: Status 404 returned error can't find the container with id 16916b0d92daf26e5ed6eb43f8926f96eb497b3deba66eca3e9329105a0793ac Jul 28 00:25:57.007926 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:57.007898 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b9a37a8a1a4f4f13a95b97af3fddb62b028474b09e30a3a02475f5f16ccf8bf" Jul 28 00:25:57.007926 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:57.007901 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:57.009094 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:57.009054 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349292032" event={"ID":"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2","Type":"ContainerStarted","Data":"16916b0d92daf26e5ed6eb43f8926f96eb497b3deba66eca3e9329105a0793ac"} Jul 28 00:25:57.018767 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:25:57.018737 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7406-139999349291024\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="17acce5c-579e-4990-9003-96960aa39ed7" pod="opendatahub/test-curl-7406-139999349291024" Jul 28 00:25:57.173396 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:57.173366 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17acce5c-579e-4990-9003-96960aa39ed7" path="/var/lib/kubelet/pods/17acce5c-579e-4990-9003-96960aa39ed7/volumes" Jul 28 00:25:58.015464 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:58.015424 2574 generic.go:356] "Generic (PLEG): container finished" podID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" containerID="3d15856e540e90438d2b619f9ec2242875ca76805cbbadb45a5c40af03ac44fc" exitCode=0 Jul 28 00:25:58.015673 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:58.015510 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349292032" event={"ID":"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2","Type":"ContainerDied","Data":"3d15856e540e90438d2b619f9ec2242875ca76805cbbadb45a5c40af03ac44fc"} Jul 28 00:25:59.159481 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.159456 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.178628 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.178597 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999349292032_76d4b5c7-9c7b-4ade-b324-b12c1ea358d2/test-curl-7406-139999349292032/0.log" Jul 28 00:25:59.202723 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.202685 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:59.205755 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.205726 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:59.317790 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.317704 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6g9z\" (UniqueName: \"kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z\") pod \"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2\" (UID: \"76d4b5c7-9c7b-4ade-b324-b12c1ea358d2\") " Jul 28 00:25:59.319843 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.319820 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z" (OuterVolumeSpecName: "kube-api-access-q6g9z") pod "76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" (UID: "76d4b5c7-9c7b-4ade-b324-b12c1ea358d2"). InnerVolumeSpecName "kube-api-access-q6g9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:25:59.335503 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.335475 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:59.335871 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.335857 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" containerName="test-curl-7406-139999349292032" Jul 28 00:25:59.335914 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.335873 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" containerName="test-curl-7406-139999349292032" Jul 28 00:25:59.335945 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.335928 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" containerName="test-curl-7406-139999349292032" Jul 28 00:25:59.338784 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.338763 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.344801 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.344772 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:59.419055 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.419016 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6g9z\" (UniqueName: \"kubernetes.io/projected/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2-kube-api-access-q6g9z\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:25:59.519634 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.519595 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7svh\" (UniqueName: \"kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh\") pod \"test-curl-7406-139999349292032\" (UID: \"ca1c9358-4576-4608-842b-dce1d1762c81\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.620673 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.620575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7svh\" (UniqueName: \"kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh\") pod \"test-curl-7406-139999349292032\" (UID: \"ca1c9358-4576-4608-842b-dce1d1762c81\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.628732 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.628696 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7svh\" (UniqueName: \"kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh\") pod \"test-curl-7406-139999349292032\" (UID: \"ca1c9358-4576-4608-842b-dce1d1762c81\") " pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.648356 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.648322 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:25:59.794055 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:25:59.794021 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:25:59.797063 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:25:59.797034 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca1c9358_4576_4608_842b_dce1d1762c81.slice/crio-c5bb951c5777fe9a641f3881dba8a2a72555db5e3571804456e563270893e4c0 WatchSource:0}: Error finding container c5bb951c5777fe9a641f3881dba8a2a72555db5e3571804456e563270893e4c0: Status 404 returned error can't find the container with id c5bb951c5777fe9a641f3881dba8a2a72555db5e3571804456e563270893e4c0 Jul 28 00:26:00.026869 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:00.026828 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349292032" event={"ID":"ca1c9358-4576-4608-842b-dce1d1762c81","Type":"ContainerStarted","Data":"c5bb951c5777fe9a641f3881dba8a2a72555db5e3571804456e563270893e4c0"} Jul 28 00:26:00.028124 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:00.028101 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16916b0d92daf26e5ed6eb43f8926f96eb497b3deba66eca3e9329105a0793ac" Jul 28 00:26:00.028124 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:00.028105 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:26:00.039918 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:00.039885 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7406-139999349292032" oldPodUID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" Jul 28 00:26:01.037458 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:01.037414 2574 generic.go:356] "Generic (PLEG): container finished" podID="ca1c9358-4576-4608-842b-dce1d1762c81" containerID="229a7bacff16d43942e14099fd82ff676f8fde03dbd00be0121664ec7f42e864" exitCode=0 Jul 28 00:26:01.037961 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:01.037553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7406-139999349292032" event={"ID":"ca1c9358-4576-4608-842b-dce1d1762c81","Type":"ContainerDied","Data":"229a7bacff16d43942e14099fd82ff676f8fde03dbd00be0121664ec7f42e864"} Jul 28 00:26:01.173367 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:01.173332 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d4b5c7-9c7b-4ade-b324-b12c1ea358d2" path="/var/lib/kubelet/pods/76d4b5c7-9c7b-4ade-b324-b12c1ea358d2/volumes" Jul 28 00:26:02.177024 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.176994 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:26:02.196372 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.196342 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7406-139999349292032_ca1c9358-4576-4608-842b-dce1d1762c81/test-curl-7406-139999349292032/0.log" Jul 28 00:26:02.219435 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.219394 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:26:02.220929 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.220908 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7406-139999349292032"] Jul 28 00:26:02.242668 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.242635 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7svh\" (UniqueName: \"kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh\") pod \"ca1c9358-4576-4608-842b-dce1d1762c81\" (UID: \"ca1c9358-4576-4608-842b-dce1d1762c81\") " Jul 28 00:26:02.244663 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.244639 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh" (OuterVolumeSpecName: "kube-api-access-r7svh") pod "ca1c9358-4576-4608-842b-dce1d1762c81" (UID: "ca1c9358-4576-4608-842b-dce1d1762c81"). InnerVolumeSpecName "kube-api-access-r7svh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:26:02.343757 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:02.343670 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7svh\" (UniqueName: \"kubernetes.io/projected/ca1c9358-4576-4608-842b-dce1d1762c81-kube-api-access-r7svh\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:26:03.050555 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:03.050525 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:26:03.050723 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:03.050526 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5bb951c5777fe9a641f3881dba8a2a72555db5e3571804456e563270893e4c0" Jul 28 00:26:03.060600 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:26:03.060569 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7406-139999349292032\" is forbidden: User \"system:node:ip-10-0-128-69.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-69.ec2.internal' and this object" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" pod="opendatahub/test-curl-7406-139999349292032" Jul 28 00:26:03.173218 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:03.173178 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" path="/var/lib/kubelet/pods/ca1c9358-4576-4608-842b-dce1d1762c81/volumes" Jul 28 00:26:14.677494 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:14.677447 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:26:32.285657 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:32.285568 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:26:59.185062 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:26:59.185017 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:27:05.297681 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:05.297654 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:27:05.301905 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:05.301883 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksj56_5cb08f9e-fb93-44ff-aee9-aa4e38b10d44/ovn-acl-logging/0.log" Jul 28 00:27:33.284024 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:33.283972 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:27:45.582943 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.582899 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:27:45.583472 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.583364 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" containerName="test-curl-7406-139999349292032" Jul 28 00:27:45.583472 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.583382 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" containerName="test-curl-7406-139999349292032" Jul 28 00:27:45.583472 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.583467 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca1c9358-4576-4608-842b-dce1d1762c81" containerName="test-curl-7406-139999349292032" Jul 28 00:27:45.586733 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.586715 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.590189 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.590161 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d194aa-openshift-default-dockercfg-42vmr\"" Jul 28 00:27:45.590368 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.590352 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 28 00:27:45.612618 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.612583 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:27:45.678134 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z889p\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678134 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678370 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678370 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678193 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678370 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678370 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678370 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678582 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.678582 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.678411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779109 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779074 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z889p\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779329 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779687 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779687 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779360 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779793 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779779 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779846 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779798 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.779925 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.779907 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.780027 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.780004 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.780238 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.780216 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.781643 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.781624 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.781869 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.781851 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.790970 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.790941 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z889p\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.791087 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.791064 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token\") pod \"e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:45.896669 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:45.896627 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:46.037719 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:46.037487 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:27:46.040064 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:27:46.040039 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2829e87_1276_4426_9ba3_d116e2f06640.slice/crio-e028f2bf01f537c05e619c796fab7c2d0d0a88a1a4a0879742371e8ca662a5cd WatchSource:0}: Error finding container e028f2bf01f537c05e619c796fab7c2d0d0a88a1a4a0879742371e8ca662a5cd: Status 404 returned error can't find the container with id e028f2bf01f537c05e619c796fab7c2d0d0a88a1a4a0879742371e8ca662a5cd Jul 28 00:27:46.573555 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:46.573489 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerStarted","Data":"e028f2bf01f537c05e619c796fab7c2d0d0a88a1a4a0879742371e8ca662a5cd"} Jul 28 00:27:49.070681 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.070639 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:27:49.070961 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.070714 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:27:49.070961 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.070745 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:27:49.598830 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.598796 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerStarted","Data":"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a"} Jul 28 00:27:49.617882 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.617826 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podStartSLOduration=1.589845156 podStartE2EDuration="4.617807815s" totalImagesPullingTime="3.027962659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:27:49.616566965 +0000 UTC m=+3044.977103645" watchObservedRunningTime="2026-07-28 00:27:49.617807815 +0000 UTC m=+3044.978344492" Jul 28 00:27:49.897770 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:49.897729 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:50.902735 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:50.902700 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:51.608173 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:51.608139 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:51.608998 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:51.608978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:27:53.338379 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.338340 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:27:53.342046 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.342025 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.344654 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.344626 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 28 00:27:53.344798 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.344700 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rpq2s\"" Jul 28 00:27:53.345554 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.345533 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 00:27:53.345554 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.345548 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d194aa-serving-cert\"" Jul 28 00:27:53.350471 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.350445 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:27:53.457365 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.457329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.457365 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.457368 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmb7\" (UniqueName: \"kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.457688 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.457424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.558986 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.558933 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.558986 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.558988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmb7\" (UniqueName: \"kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.559205 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.559023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.559205 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:27:53.559080 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-serving-cert: secret "maas-api-e2e-ipp-cleanup-d194aa-serving-cert" not found Jul 28 00:27:53.559205 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:27:53.559148 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls podName:a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e nodeName:}" failed. No retries permitted until 2026-07-28 00:27:54.059130834 +0000 UTC m=+3049.419667490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" (UID: "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e") : secret "maas-api-e2e-ipp-cleanup-d194aa-serving-cert" not found Jul 28 00:27:53.559756 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.559738 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.567593 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.567570 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmb7\" (UniqueName: \"kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:53.972735 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.972691 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:27:53.976421 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.976396 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:53.978956 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.978931 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d194aa\"" Jul 28 00:27:53.985791 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:53.985764 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:27:54.030993 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.030948 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:27:54.036690 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.036659 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:27:54.036851 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.036766 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.039315 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.039291 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d194aa-dockercfg-hj7qc\"" Jul 28 00:27:54.063890 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.063855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc65c\" (UniqueName: \"kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.064077 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.063910 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:54.064077 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.063985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.066445 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.066422 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:54.164749 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.164709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc65c\" (UniqueName: \"kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.164942 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.164763 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwqt\" (UniqueName: \"kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.164942 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.164809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.164942 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.164837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.165554 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.165535 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.172386 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.172358 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc65c\" (UniqueName: \"kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c\") pod \"payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.254255 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.254139 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:54.265766 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.265733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.265913 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.265863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwqt\" (UniqueName: \"kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.266479 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.266456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.274609 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.274579 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwqt\" (UniqueName: \"kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt\") pod \"payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.288161 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.288125 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:54.350029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.349870 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:54.410280 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.410251 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:27:54.413705 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:27:54.413654 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c287bb_86f1_45e6_bf2f_5371ae4b6a2e.slice/crio-5b53c0a29a55c3a5b39fbe873ab7bde391508c9a68035271c305d191fd0109f0 WatchSource:0}: Error finding container 5b53c0a29a55c3a5b39fbe873ab7bde391508c9a68035271c305d191fd0109f0: Status 404 returned error can't find the container with id 5b53c0a29a55c3a5b39fbe873ab7bde391508c9a68035271c305d191fd0109f0 Jul 28 00:27:54.464015 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.463907 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:27:54.466859 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:27:54.466820 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32147c30_38ee_47c3_b0fc_7bfcc449c281.slice/crio-f983aa4a61fa7d7b8517cc0472fbd56de606eee77d8155b41251ef1a0b31700c WatchSource:0}: Error finding container f983aa4a61fa7d7b8517cc0472fbd56de606eee77d8155b41251ef1a0b31700c: Status 404 returned error can't find the container with id f983aa4a61fa7d7b8517cc0472fbd56de606eee77d8155b41251ef1a0b31700c Jul 28 00:27:54.511465 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.511438 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:27:54.513320 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:27:54.513292 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9e9a02_d95a_496e_898e_1c0b6f130346.slice/crio-78eaf92523b468472f0bb5decc3823fa2796171209bbcf4abbb8cf6e32e6a56e WatchSource:0}: Error finding container 78eaf92523b468472f0bb5decc3823fa2796171209bbcf4abbb8cf6e32e6a56e: Status 404 returned error can't find the container with id 78eaf92523b468472f0bb5decc3823fa2796171209bbcf4abbb8cf6e32e6a56e Jul 28 00:27:54.624433 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.624387 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" event={"ID":"32147c30-38ee-47c3-b0fc-7bfcc449c281","Type":"ContainerStarted","Data":"f983aa4a61fa7d7b8517cc0472fbd56de606eee77d8155b41251ef1a0b31700c"} Jul 28 00:27:54.625550 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.625494 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" event={"ID":"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e","Type":"ContainerStarted","Data":"5b53c0a29a55c3a5b39fbe873ab7bde391508c9a68035271c305d191fd0109f0"} Jul 28 00:27:54.626806 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:54.626781 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" event={"ID":"ab9e9a02-d95a-496e-898e-1c0b6f130346","Type":"ContainerStarted","Data":"78eaf92523b468472f0bb5decc3823fa2796171209bbcf4abbb8cf6e32e6a56e"} Jul 28 00:27:57.654759 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.654446 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" event={"ID":"ab9e9a02-d95a-496e-898e-1c0b6f130346","Type":"ContainerStarted","Data":"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2"} Jul 28 00:27:57.654759 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.654550 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:27:57.656109 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.656059 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" event={"ID":"32147c30-38ee-47c3-b0fc-7bfcc449c281","Type":"ContainerStarted","Data":"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f"} Jul 28 00:27:57.656251 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.656117 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:27:57.657633 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.657614 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" event={"ID":"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e","Type":"ContainerStarted","Data":"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d"} Jul 28 00:27:57.657740 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.657729 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:27:57.677246 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.677188 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" podStartSLOduration=0.646760432 podStartE2EDuration="3.677170641s" totalImagesPullingTime="3.030410209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:27:57.675939186 +0000 UTC m=+3053.036475865" watchObservedRunningTime="2026-07-28 00:27:57.677170641 +0000 UTC m=+3053.037707298" Jul 28 00:27:57.692583 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.692530 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" podStartSLOduration=1.550505418 podStartE2EDuration="4.6924991s" totalImagesPullingTime="3.141993682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:27:57.69135898 +0000 UTC m=+3053.051895678" watchObservedRunningTime="2026-07-28 00:27:57.6924991 +0000 UTC m=+3053.053035778" Jul 28 00:27:57.722054 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:27:57.721978 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" podStartSLOduration=1.644833558 podStartE2EDuration="4.721954923s" totalImagesPullingTime="3.077121365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:27:57.719342349 +0000 UTC m=+3053.079879029" watchObservedRunningTime="2026-07-28 00:27:57.721954923 +0000 UTC m=+3053.082491604" Jul 28 00:28:01.091027 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:01.090979 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:28:03.668617 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:03.668586 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:28:07.649749 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:07.649697 2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:28:07.650227 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:07.649780 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:28:08.127535 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.127430 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:08.131655 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.131630 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:08.133941 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.133916 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tpsvv\"" Jul 28 00:28:08.138366 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.138341 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:08.302178 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.302136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58dzw\" (UniqueName: \"kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw\") pod \"kuadrant-operator-catalog-xpcn7\" (UID: \"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219\") " pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:08.402885 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.402848 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58dzw\" (UniqueName: \"kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw\") pod \"kuadrant-operator-catalog-xpcn7\" (UID: \"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219\") " pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:08.411289 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.411261 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58dzw\" (UniqueName: \"kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw\") pod \"kuadrant-operator-catalog-xpcn7\" (UID: \"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219\") " pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:08.444480 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.444444 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:08.584342 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.584304 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:08.586747 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:28:08.586715 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289aa0b0_66b3_4dfc_bba6_c0d1b7d31219.slice/crio-bdecb8c7957b825144fc0012fd218bcf231ebac6a98adacf316a51017f578a59 WatchSource:0}: Error finding container bdecb8c7957b825144fc0012fd218bcf231ebac6a98adacf316a51017f578a59: Status 404 returned error can't find the container with id bdecb8c7957b825144fc0012fd218bcf231ebac6a98adacf316a51017f578a59 Jul 28 00:28:08.665267 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.665187 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:28:08.665642 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.665287 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:28:08.714821 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:08.714782 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" event={"ID":"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219","Type":"ContainerStarted","Data":"bdecb8c7957b825144fc0012fd218bcf231ebac6a98adacf316a51017f578a59"} Jul 28 00:28:09.722868 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:09.722788 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2829e87-1276-4426-9ba3-d116e2f06640" containerID="989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a" exitCode=0 Jul 28 00:28:09.723380 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:09.722874 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerDied","Data":"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a"} Jul 28 00:28:09.723447 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:09.723392 2574 scope.go:117] "RemoveContainer" containerID="989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a" Jul 28 00:28:10.017071 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.017017 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:28:10.017262 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.017116 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:28:10.017262 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.017170 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:28:10.732804 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.732746 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" event={"ID":"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219","Type":"ContainerStarted","Data":"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0"} Jul 28 00:28:10.739166 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.739124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerStarted","Data":"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890"} Jul 28 00:28:10.755795 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.755727 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" podStartSLOduration=1.2762801160000001 podStartE2EDuration="2.755708992s" totalImagesPullingTime="1.479428876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:28:10.752957544 +0000 UTC m=+3066.113494223" watchObservedRunningTime="2026-07-28 00:28:10.755708992 +0000 UTC m=+3066.116245670" Jul 28 00:28:10.897115 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:10.897080 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:11.897410 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:11.897372 2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:28:11.897848 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:11.897429 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 28 00:28:12.897241 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:12.897205 2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:28:12.897425 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:12.897262 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 28 00:28:13.897845 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:13.897803 2574 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:28:13.898252 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:13.897862 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 28 00:28:14.900952 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:14.900914 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:14.901354 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:14.901289 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:14.902048 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:14.902024 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:18.444694 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:18.444642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:18.445126 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:18.444719 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:18.467966 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:18.467938 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:18.806765 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:18.806683 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:18.841354 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:18.841314 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:20.794413 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:20.794374 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" podUID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" containerName="registry-server" containerID="cri-o://bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0" gracePeriod=2 Jul 28 00:28:21.043633 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.043603 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:21.123633 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.123534 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58dzw\" (UniqueName: \"kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw\") pod \"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219\" (UID: \"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219\") " Jul 28 00:28:21.125765 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.125733 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw" (OuterVolumeSpecName: "kube-api-access-58dzw") pod "289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" (UID: "289aa0b0-66b3-4dfc-bba6-c0d1b7d31219"). InnerVolumeSpecName "kube-api-access-58dzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:21.224315 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.224277 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58dzw\" (UniqueName: \"kubernetes.io/projected/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219-kube-api-access-58dzw\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:21.800715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.800676 2574 generic.go:356] "Generic (PLEG): container finished" podID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" containerID="bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0" exitCode=0 Jul 28 00:28:21.800715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.800716 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" Jul 28 00:28:21.801175 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.800754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" event={"ID":"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219","Type":"ContainerDied","Data":"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0"} Jul 28 00:28:21.801175 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.800812 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xpcn7" event={"ID":"289aa0b0-66b3-4dfc-bba6-c0d1b7d31219","Type":"ContainerDied","Data":"bdecb8c7957b825144fc0012fd218bcf231ebac6a98adacf316a51017f578a59"} Jul 28 00:28:21.801175 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.800834 2574 scope.go:117] "RemoveContainer" containerID="bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0" Jul 28 00:28:21.815464 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.815443 2574 scope.go:117] "RemoveContainer" containerID="bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0" Jul 28 00:28:21.815781 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:21.815757 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0\": container with ID starting with bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0 not found: ID does not exist" containerID="bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0" Jul 28 00:28:21.815836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.815789 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0"} err="failed to get container status \"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0\": rpc error: code = NotFound desc = could not find container \"bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0\": container with ID starting with bb7041534ed1d7842076eb874b7d103bae06c03bdb6b6dcc10d9aeeda49a22c0 not found: ID does not exist" Jul 28 00:28:21.819598 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.819572 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:21.824936 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:21.824911 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xpcn7"] Jul 28 00:28:23.122566 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.122495 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:28:23.123018 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.122823 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" podUID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" containerName="maas-api" containerID="cri-o://5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d" gracePeriod=30 Jul 28 00:28:23.164505 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.164464 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:28:23.164817 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.164758 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" podUID="ab9e9a02-d95a-496e-898e-1c0b6f130346" containerName="payload-processing" containerID="cri-o://c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2" gracePeriod=30 Jul 28 00:28:23.177844 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.177804 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" path="/var/lib/kubelet/pods/289aa0b0-66b3-4dfc-bba6-c0d1b7d31219/volumes" Jul 28 00:28:23.178171 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.178150 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:28:23.178392 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.178359 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" podUID="32147c30-38ee-47c3-b0fc-7bfcc449c281" containerName="payload-pre-processing" containerID="cri-o://cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f" gracePeriod=30 Jul 28 00:28:23.457041 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.457014 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:28:23.460389 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.460360 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:28:23.488585 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.488550 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:28:23.543215 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543186 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca\") pod \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " Jul 28 00:28:23.543401 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543223 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") pod \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " Jul 28 00:28:23.543401 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543258 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume\") pod \"ab9e9a02-d95a-496e-898e-1c0b6f130346\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " Jul 28 00:28:23.543401 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543282 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzmb7\" (UniqueName: \"kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7\") pod \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\" (UID: \"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e\") " Jul 28 00:28:23.543401 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543345 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stwqt\" (UniqueName: \"kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt\") pod \"ab9e9a02-d95a-496e-898e-1c0b6f130346\" (UID: \"ab9e9a02-d95a-496e-898e-1c0b6f130346\") " Jul 28 00:28:23.543401 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543392 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume\") pod \"32147c30-38ee-47c3-b0fc-7bfcc449c281\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " Jul 28 00:28:23.543774 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543438 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc65c\" (UniqueName: \"kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c\") pod \"32147c30-38ee-47c3-b0fc-7bfcc449c281\" (UID: \"32147c30-38ee-47c3-b0fc-7bfcc449c281\") " Jul 28 00:28:23.543774 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543706 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab9e9a02-d95a-496e-898e-1c0b6f130346" (UID: "ab9e9a02-d95a-496e-898e-1c0b6f130346"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:28:23.543884 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543807 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32147c30-38ee-47c3-b0fc-7bfcc449c281" (UID: "32147c30-38ee-47c3-b0fc-7bfcc449c281"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:28:23.543971 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.543949 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" (UID: "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:28:23.545925 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.545884 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt" (OuterVolumeSpecName: "kube-api-access-stwqt") pod "ab9e9a02-d95a-496e-898e-1c0b6f130346" (UID: "ab9e9a02-d95a-496e-898e-1c0b6f130346"). InnerVolumeSpecName "kube-api-access-stwqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:23.546066 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.546047 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" (UID: "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 00:28:23.546128 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.546087 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7" (OuterVolumeSpecName: "kube-api-access-lzmb7") pod "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" (UID: "a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e"). InnerVolumeSpecName "kube-api-access-lzmb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:23.546128 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.546096 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c" (OuterVolumeSpecName: "kube-api-access-pc65c") pod "32147c30-38ee-47c3-b0fc-7bfcc449c281" (UID: "32147c30-38ee-47c3-b0fc-7bfcc449c281"). InnerVolumeSpecName "kube-api-access-pc65c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:23.644752 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644710 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32147c30-38ee-47c3-b0fc-7bfcc449c281-plugins-config-volume\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644752 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644744 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc65c\" (UniqueName: \"kubernetes.io/projected/32147c30-38ee-47c3-b0fc-7bfcc449c281-kube-api-access-pc65c\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644752 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644755 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-openshift-service-ca\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644984 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644764 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-maas-api-tls\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644984 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644772 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab9e9a02-d95a-496e-898e-1c0b6f130346-plugins-config-volume\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644984 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644781 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzmb7\" (UniqueName: \"kubernetes.io/projected/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e-kube-api-access-lzmb7\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.644984 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.644790 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stwqt\" (UniqueName: \"kubernetes.io/projected/ab9e9a02-d95a-496e-898e-1c0b6f130346-kube-api-access-stwqt\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:23.813649 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.813618 2574 generic.go:356] "Generic (PLEG): container finished" podID="32147c30-38ee-47c3-b0fc-7bfcc449c281" containerID="cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f" exitCode=0 Jul 28 00:28:23.813836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.813661 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" Jul 28 00:28:23.813836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.813695 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" event={"ID":"32147c30-38ee-47c3-b0fc-7bfcc449c281","Type":"ContainerDied","Data":"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f"} Jul 28 00:28:23.813836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.813741 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk" event={"ID":"32147c30-38ee-47c3-b0fc-7bfcc449c281","Type":"ContainerDied","Data":"f983aa4a61fa7d7b8517cc0472fbd56de606eee77d8155b41251ef1a0b31700c"} Jul 28 00:28:23.813836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.813769 2574 scope.go:117] "RemoveContainer" containerID="cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f" Jul 28 00:28:23.815169 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.815145 2574 generic.go:356] "Generic (PLEG): container finished" podID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" containerID="5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d" exitCode=0 Jul 28 00:28:23.815282 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.815165 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" event={"ID":"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e","Type":"ContainerDied","Data":"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d"} Jul 28 00:28:23.815282 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.815195 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" Jul 28 00:28:23.815282 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.815214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7" event={"ID":"a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e","Type":"ContainerDied","Data":"5b53c0a29a55c3a5b39fbe873ab7bde391508c9a68035271c305d191fd0109f0"} Jul 28 00:28:23.816577 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.816551 2574 generic.go:356] "Generic (PLEG): container finished" podID="ab9e9a02-d95a-496e-898e-1c0b6f130346" containerID="c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2" exitCode=0 Jul 28 00:28:23.816684 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.816640 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" Jul 28 00:28:23.816684 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.816643 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" event={"ID":"ab9e9a02-d95a-496e-898e-1c0b6f130346","Type":"ContainerDied","Data":"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2"} Jul 28 00:28:23.816801 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.816699 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8" event={"ID":"ab9e9a02-d95a-496e-898e-1c0b6f130346","Type":"ContainerDied","Data":"78eaf92523b468472f0bb5decc3823fa2796171209bbcf4abbb8cf6e32e6a56e"} Jul 28 00:28:23.825106 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.825083 2574 scope.go:117] "RemoveContainer" containerID="cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f" Jul 28 00:28:23.825435 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:23.825409 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f\": container with ID starting with cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f not found: ID does not exist" containerID="cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f" Jul 28 00:28:23.825570 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.825445 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f"} err="failed to get container status \"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f\": rpc error: code = NotFound desc = could not find container \"cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f\": container with ID starting with cd84d7203ab4d3431cd8db2bfcbd4a216cf2b0bda281b05d8acd3bd02b47dc0f not found: ID does not exist" Jul 28 00:28:23.825570 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.825466 2574 scope.go:117] "RemoveContainer" containerID="5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d" Jul 28 00:28:23.834243 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.834226 2574 scope.go:117] "RemoveContainer" containerID="5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d" Jul 28 00:28:23.834501 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:23.834482 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d\": container with ID starting with 5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d not found: ID does not exist" containerID="5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d" Jul 28 00:28:23.834595 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.834509 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d"} err="failed to get container status \"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d\": rpc error: code = NotFound desc = could not find container \"5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d\": container with ID starting with 5cc5fb1546c349ab513f56b7ba2a6181daf150d295b43204624e530af5191d9d not found: ID does not exist" Jul 28 00:28:23.834595 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.834541 2574 scope.go:117] "RemoveContainer" containerID="c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2" Jul 28 00:28:23.840072 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.840040 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:28:23.844225 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.844199 2574 scope.go:117] "RemoveContainer" containerID="c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2" Jul 28 00:28:23.844547 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:23.844501 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2\": container with ID starting with c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2 not found: ID does not exist" containerID="c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2" Jul 28 00:28:23.844624 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.844542 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2"} err="failed to get container status \"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2\": rpc error: code = NotFound desc = could not find container \"c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2\": container with ID starting with c370e785a69e2b0b1860f3bd26a495d86ec4b6badd990bc8a824711ad4b6abb2 not found: ID does not exist" Jul 28 00:28:23.844785 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.844767 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d194aa-d7c8647c6-vvcxk"] Jul 28 00:28:23.853015 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.852991 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:28:23.856441 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.856419 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d194aa-79b6859846-9jsz7"] Jul 28 00:28:23.865759 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.865732 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:28:23.870997 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:23.870975 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d194aa-7864455dfb-pf7l8"] Jul 28 00:28:25.172907 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:25.172870 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32147c30-38ee-47c3-b0fc-7bfcc449c281" path="/var/lib/kubelet/pods/32147c30-38ee-47c3-b0fc-7bfcc449c281/volumes" Jul 28 00:28:25.173421 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:25.173331 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" path="/var/lib/kubelet/pods/a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e/volumes" Jul 28 00:28:25.173805 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:25.173782 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9e9a02-d95a-496e-898e-1c0b6f130346" path="/var/lib/kubelet/pods/ab9e9a02-d95a-496e-898e-1c0b6f130346/volumes" Jul 28 00:28:29.485391 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:29.485350 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:28:33.911648 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:33.911601 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:28:33.912163 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:33.911845 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" containerID="cri-o://edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890" gracePeriod=30 Jul 28 00:28:39.175947 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.175883 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:39.293859 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.293816 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294065 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.293917 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294065 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.293955 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294065 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.293981 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294065 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294023 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294065 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294056 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294325 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294093 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294325 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294122 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294325 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294192 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z889p\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p\") pod \"f2829e87-1276-4426-9ba3-d116e2f06640\" (UID: \"f2829e87-1276-4426-9ba3-d116e2f06640\") " Jul 28 00:28:39.294474 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294344 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:28:39.294474 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294359 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:28:39.294615 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294580 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2829e87-1276-4426-9ba3-d116e2f06640-istiod-ca-cert\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.294615 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294603 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-credential-socket\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.294709 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294613 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:28:39.294814 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294794 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:28:39.294873 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.294854 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:28:39.296347 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.296324 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:39.296422 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.296397 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p" (OuterVolumeSpecName: "kube-api-access-z889p") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "kube-api-access-z889p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:28:39.296573 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.296551 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:28:39.296636 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.296569 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2829e87-1276-4426-9ba3-d116e2f06640" (UID: "f2829e87-1276-4426-9ba3-d116e2f06640"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:28:39.395715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395672 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-socket\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395710 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-istio-token\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395715 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395723 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-workload-certs\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395967 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395735 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z889p\" (UniqueName: \"kubernetes.io/projected/f2829e87-1276-4426-9ba3-d116e2f06640-kube-api-access-z889p\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395967 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395747 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2829e87-1276-4426-9ba3-d116e2f06640-istio-podinfo\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395967 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395757 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-envoy\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.395967 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.395767 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2829e87-1276-4426-9ba3-d116e2f06640-istio-data\") on node \"ip-10-0-128-69.ec2.internal\" DevicePath \"\"" Jul 28 00:28:39.902657 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.902615 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2829e87-1276-4426-9ba3-d116e2f06640" containerID="edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890" exitCode=0 Jul 28 00:28:39.902837 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.902665 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" Jul 28 00:28:39.902837 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.902700 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerDied","Data":"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890"} Jul 28 00:28:39.902837 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.902761 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf" event={"ID":"f2829e87-1276-4426-9ba3-d116e2f06640","Type":"ContainerDied","Data":"e028f2bf01f537c05e619c796fab7c2d0d0a88a1a4a0879742371e8ca662a5cd"} Jul 28 00:28:39.902837 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.902784 2574 scope.go:117] "RemoveContainer" containerID="edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890" Jul 28 00:28:39.917962 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.917940 2574 scope.go:117] "RemoveContainer" containerID="989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a" Jul 28 00:28:39.926937 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.926910 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:28:39.927601 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.927581 2574 scope.go:117] "RemoveContainer" containerID="edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890" Jul 28 00:28:39.927912 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:39.927891 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890\": container with ID starting with edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890 not found: ID does not exist" containerID="edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890" Jul 28 00:28:39.927977 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.927921 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890"} err="failed to get container status \"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890\": rpc error: code = NotFound desc = could not find container \"edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890\": container with ID starting with edf50dae9450cded6207d0abf9d27dd85e2e7b7589f4c7858899ae4cf143f890 not found: ID does not exist" Jul 28 00:28:39.927977 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.927940 2574 scope.go:117] "RemoveContainer" containerID="989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a" Jul 28 00:28:39.928184 ip-10-0-128-69 kubenswrapper[2574]: E0728 00:28:39.928170 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a\": container with ID starting with 989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a not found: ID does not exist" containerID="989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a" Jul 28 00:28:39.928225 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.928186 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a"} err="failed to get container status \"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a\": rpc error: code = NotFound desc = could not find container \"989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a\": container with ID starting with 989ec3c20b4832d947fd0c689c8e7af31bae1fc361523d6963c60dc7e2f2c49a not found: ID does not exist" Jul 28 00:28:39.931838 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:39.931817 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d194aa-openshift-default-fb5f85644-6dsxf"] Jul 28 00:28:41.174120 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:41.174084 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" path="/var/lib/kubelet/pods/f2829e87-1276-4426-9ba3-d116e2f06640/volumes" Jul 28 00:28:53.588605 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:28:53.588564 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:29:04.779162 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:29:04.779125 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:30:25.377927 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:30:25.377885 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:30:45.981930 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:30:45.981895 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:31:07.874949 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:07.874912 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wp5vp"] Jul 28 00:31:16.904205 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:16.904165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c956cb97-bskpx_2594fedd-1cfd-4885-bcda-31eb5e900d33/authorino/0.log" Jul 28 00:31:23.305548 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:23.305495 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-4mvj6_51401b9e-3160-46e9-b807-9ee83bd55dbf/copy-manifests/0.log" Jul 28 00:31:23.313093 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:23.313064 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-4mvj6_51401b9e-3160-46e9-b807-9ee83bd55dbf/manager/0.log" Jul 28 00:31:25.623836 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:25.623804 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c956cb97-bskpx_2594fedd-1cfd-4885-bcda-31eb5e900d33/authorino/0.log" Jul 28 00:31:25.968049 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:25.968020 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nkhnk_cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b/kuadrant-console-plugin/0.log" Jul 28 00:31:26.223913 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:26.223837 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jlmdw_925ffbce-aa7e-4905-9b5e-639429fcbbc7/manager/0.log" Jul 28 00:31:26.348421 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:26.348394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wp5vp_e5333281-b4b7-462f-aec7-e5a711662a18/limitador/0.log" Jul 28 00:31:27.101020 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:27.100982 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f44c56477-4xj82_57d2bec0-e711-4f92-bbff-9c1a8ecb8da7/kube-auth-proxy/0.log" Jul 28 00:31:37.643772 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:37.643740 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x7j27_2c884103-50ba-403b-89eb-66d666438a5c/global-pull-secret-syncer/0.log" Jul 28 00:31:37.740346 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:37.740314 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rm6tp_bd1e47f9-2f7a-4523-92f3-437b99a9f63a/konnectivity-agent/0.log" Jul 28 00:31:37.796450 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:37.796423 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-69.ec2.internal_6a62480b743c193092f01b29ef1c2800/haproxy/0.log" Jul 28 00:31:42.914878 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:42.914845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c956cb97-bskpx_2594fedd-1cfd-4885-bcda-31eb5e900d33/authorino/0.log" Jul 28 00:31:42.993852 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:42.993778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nkhnk_cc9a7bbf-d2d1-44f3-8c0f-b345081ac53b/kuadrant-console-plugin/0.log" Jul 28 00:31:43.341045 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:43.340966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jlmdw_925ffbce-aa7e-4905-9b5e-639429fcbbc7/manager/0.log" Jul 28 00:31:43.361109 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:43.361077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wp5vp_e5333281-b4b7-462f-aec7-e5a711662a18/limitador/0.log" Jul 28 00:31:45.165924 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.165893 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/alertmanager/0.log" Jul 28 00:31:45.186721 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.186695 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/config-reloader/0.log" Jul 28 00:31:45.212472 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.212450 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/kube-rbac-proxy-web/0.log" Jul 28 00:31:45.237145 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.237126 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/kube-rbac-proxy/0.log" Jul 28 00:31:45.261277 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.261253 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/kube-rbac-proxy-metric/0.log" Jul 28 00:31:45.287970 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.287949 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/prom-label-proxy/0.log" Jul 28 00:31:45.306322 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.306298 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_beb23daf-dbc2-49b5-9767-675995599626/init-config-reloader/0.log" Jul 28 00:31:45.376365 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.376344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-94hps_57ef9b32-ce47-481b-b23d-550c82bcc50c/kube-state-metrics/0.log" Jul 28 00:31:45.393844 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.393823 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-94hps_57ef9b32-ce47-481b-b23d-550c82bcc50c/kube-rbac-proxy-main/0.log" Jul 28 00:31:45.415604 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.415579 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-94hps_57ef9b32-ce47-481b-b23d-550c82bcc50c/kube-rbac-proxy-self/0.log" Jul 28 00:31:45.446085 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.446015 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55669fdbd7-f7wmg_da868074-54b4-4793-bc8e-84734837af1c/metrics-server/0.log" Jul 28 00:31:45.471310 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.471290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-2zgqm_bacf92b3-e95d-4309-9b5a-3dd638203d08/monitoring-plugin/0.log" Jul 28 00:31:45.501366 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.501347 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nkcxr_f84fff22-f1c7-471d-be60-0baf1ca651a6/node-exporter/0.log" Jul 28 00:31:45.520697 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.520662 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nkcxr_f84fff22-f1c7-471d-be60-0baf1ca651a6/kube-rbac-proxy/0.log" Jul 28 00:31:45.542608 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.542593 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nkcxr_f84fff22-f1c7-471d-be60-0baf1ca651a6/init-textfile/0.log" Jul 28 00:31:45.715265 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.715180 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dl57h_9293afef-0bec-450f-a54c-37ee3bbb889c/kube-rbac-proxy-main/0.log" Jul 28 00:31:45.737851 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.737830 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dl57h_9293afef-0bec-450f-a54c-37ee3bbb889c/kube-rbac-proxy-self/0.log" Jul 28 00:31:45.760835 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.760811 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dl57h_9293afef-0bec-450f-a54c-37ee3bbb889c/openshift-state-metrics/0.log" Jul 28 00:31:45.796024 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.796006 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/prometheus/0.log" Jul 28 00:31:45.815004 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.814979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/config-reloader/0.log" Jul 28 00:31:45.835024 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.835004 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/thanos-sidecar/0.log" Jul 28 00:31:45.855279 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.855254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/kube-rbac-proxy-web/0.log" Jul 28 00:31:45.880689 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.880672 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/kube-rbac-proxy/0.log" Jul 28 00:31:45.901067 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.901047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/kube-rbac-proxy-thanos/0.log" Jul 28 00:31:45.920999 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.920979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a584638-ca00-4381-a778-649abf5141b2/init-config-reloader/0.log" Jul 28 00:31:45.949823 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.949806 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ksvrv_ed1681b0-306c-4ff1-a304-54a1f2aa3cd7/prometheus-operator/0.log" Jul 28 00:31:45.967677 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.967617 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ksvrv_ed1681b0-306c-4ff1-a304-54a1f2aa3cd7/kube-rbac-proxy/0.log" Jul 28 00:31:45.990901 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:45.990881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-w6tmq_e0890dc4-52fc-4e40-a36f-4ba2b4651f56/prometheus-operator-admission-webhook/0.log" Jul 28 00:31:46.094916 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.094894 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/thanos-query/0.log" Jul 28 00:31:46.114494 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.114475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/kube-rbac-proxy-web/0.log" Jul 28 00:31:46.134019 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.133985 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/kube-rbac-proxy/0.log" Jul 28 00:31:46.159662 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.159641 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/prom-label-proxy/0.log" Jul 28 00:31:46.189441 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.189423 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/kube-rbac-proxy-rules/0.log" Jul 28 00:31:46.209532 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:46.209496 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d66f84d-2s9vf_b5195fb0-43a0-4373-a1f9-68a8a504d511/kube-rbac-proxy-metrics/0.log" Jul 28 00:31:47.735653 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.735618 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5"] Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.735961 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32147c30-38ee-47c3-b0fc-7bfcc449c281" containerName="payload-pre-processing" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.735972 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="32147c30-38ee-47c3-b0fc-7bfcc449c281" containerName="payload-pre-processing" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.735989 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" containerName="registry-server" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.735995 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" containerName="registry-server" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736002 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9e9a02-d95a-496e-898e-1c0b6f130346" containerName="payload-processing" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736008 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9e9a02-d95a-496e-898e-1c0b6f130346" containerName="payload-processing" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736027 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736029 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736032 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736040 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736045 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736056 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" containerName="maas-api" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736061 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" containerName="maas-api" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736113 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736122 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="32147c30-38ee-47c3-b0fc-7bfcc449c281" containerName="payload-pre-processing" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736129 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2829e87-1276-4426-9ba3-d116e2f06640" containerName="istio-proxy" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736136 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7c287bb-86f1-45e6-bf2f-5371ae4b6a2e" containerName="maas-api" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736141 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9e9a02-d95a-496e-898e-1c0b6f130346" containerName="payload-processing" Jul 28 00:31:47.736276 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.736147 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="289aa0b0-66b3-4dfc-bba6-c0d1b7d31219" containerName="registry-server" Jul 28 00:31:47.738674 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.738658 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.741194 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.741177 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mjdrz\"/\"default-dockercfg-m9kzq\"" Jul 28 00:31:47.741287 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.741251 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mjdrz\"/\"kube-root-ca.crt\"" Jul 28 00:31:47.742195 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.742179 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mjdrz\"/\"openshift-service-ca.crt\"" Jul 28 00:31:47.748306 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.748287 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5"] Jul 28 00:31:47.781499 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.781474 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-lib-modules\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.781627 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.781507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mkb\" (UniqueName: \"kubernetes.io/projected/708a7525-3b64-4fad-a69e-e8567404d03c-kube-api-access-t4mkb\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.781627 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.781552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-podres\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.781627 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.781577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-proc\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.781732 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.781651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-sys\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882407 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882381 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-lib-modules\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882418 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mkb\" (UniqueName: \"kubernetes.io/projected/708a7525-3b64-4fad-a69e-e8567404d03c-kube-api-access-t4mkb\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-podres\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882470 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-proc\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882508 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-sys\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882776 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882568 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-lib-modules\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882776 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882608 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-proc\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882776 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882609 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-sys\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.882776 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.882633 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/708a7525-3b64-4fad-a69e-e8567404d03c-podres\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:47.890167 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:47.890151 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mkb\" (UniqueName: \"kubernetes.io/projected/708a7525-3b64-4fad-a69e-e8567404d03c-kube-api-access-t4mkb\") pod \"perf-node-gather-daemonset-tdqs5\" (UID: \"708a7525-3b64-4fad-a69e-e8567404d03c\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:48.048610 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.048507 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:48.169913 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.169800 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5"] Jul 28 00:31:48.172593 ip-10-0-128-69 kubenswrapper[2574]: W0728 00:31:48.172558 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod708a7525_3b64_4fad_a69e_e8567404d03c.slice/crio-755c2eab6ef1f141c6aa05c3e5b820c4bcc3cd55976ae55ce3504d03cd4583ad WatchSource:0}: Error finding container 755c2eab6ef1f141c6aa05c3e5b820c4bcc3cd55976ae55ce3504d03cd4583ad: Status 404 returned error can't find the container with id 755c2eab6ef1f141c6aa05c3e5b820c4bcc3cd55976ae55ce3504d03cd4583ad Jul 28 00:31:48.502381 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.502354 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68cffd5c54-4672h_b64dc2c3-9b55-4d99-9485-3767eff15c0a/console/0.log" Jul 28 00:31:48.850537 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.850425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" event={"ID":"708a7525-3b64-4fad-a69e-e8567404d03c","Type":"ContainerStarted","Data":"e0aecb38e14c1d47c578454b5869dcd7ed532167e2cc151639f9773eb3e44f12"} Jul 28 00:31:48.850537 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.850484 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" event={"ID":"708a7525-3b64-4fad-a69e-e8567404d03c","Type":"ContainerStarted","Data":"755c2eab6ef1f141c6aa05c3e5b820c4bcc3cd55976ae55ce3504d03cd4583ad"} Jul 28 00:31:48.850537 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.850523 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:48.866717 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:48.866660 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" podStartSLOduration=1.866644535 podStartE2EDuration="1.866644535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:31:48.864713863 +0000 UTC m=+3284.225250575" watchObservedRunningTime="2026-07-28 00:31:48.866644535 +0000 UTC m=+3284.227181210" Jul 28 00:31:50.180170 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:50.180140 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9xjf_ee1e08ab-c13b-4544-a1b9-b420f86b9670/dns/0.log" Jul 28 00:31:50.200853 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:50.200828 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9xjf_ee1e08ab-c13b-4544-a1b9-b420f86b9670/kube-rbac-proxy/0.log" Jul 28 00:31:50.247795 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:50.247769 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xd4ck_25a22761-c48a-40c9-a7b7-5206184e6615/dns-node-resolver/0.log" Jul 28 00:31:50.865456 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:50.865423 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rhcb4_6e6dfd4f-793c-477c-8bde-43d14d6e1ac6/node-ca/0.log" Jul 28 00:31:51.959188 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:51.959163 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f44c56477-4xj82_57d2bec0-e711-4f92-bbff-9c1a8ecb8da7/kube-auth-proxy/0.log" Jul 28 00:31:52.936579 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:52.936552 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4xlw5_6ea3a782-5d0f-4ec5-aeb6-8e54c1fe083c/serve-healthcheck-canary/0.log" Jul 28 00:31:53.527223 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:53.527199 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mfhf_eee4bc17-cf15-4918-b9e9-96e58bf78871/kube-rbac-proxy/0.log" Jul 28 00:31:53.546080 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:53.546058 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mfhf_eee4bc17-cf15-4918-b9e9-96e58bf78871/exporter/0.log" Jul 28 00:31:53.566606 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:53.566584 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mfhf_eee4bc17-cf15-4918-b9e9-96e58bf78871/extractor/0.log" Jul 28 00:31:54.864255 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:54.864223 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-tdqs5" Jul 28 00:31:56.470800 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:56.470774 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-4mvj6_51401b9e-3160-46e9-b807-9ee83bd55dbf/manager/0.log" Jul 28 00:31:56.509565 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:56.509547 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-4mvj6_51401b9e-3160-46e9-b807-9ee83bd55dbf/copy-manifests/0.log" Jul 28 00:31:57.980840 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:57.980809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65c89cb49-lcs4h_165bbeed-e12c-4ed2-b1bd-73c8ff78d64f/manager/0.log" Jul 28 00:31:58.031068 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:58.031039 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-86fnq_82674fad-05a0-4b4c-ad8b-2bca7b8c4b73/openshift-lws-operator/0.log" Jul 28 00:31:58.211275 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:31:58.211193 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f44c56477-4xj82_57d2bec0-e711-4f92-bbff-9c1a8ecb8da7/kube-auth-proxy/0.log" Jul 28 00:32:00.115553 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:00.115496 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f44c56477-4xj82_57d2bec0-e711-4f92-bbff-9c1a8ecb8da7/kube-auth-proxy/0.log" Jul 28 00:32:04.617180 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.617148 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/kube-multus-additional-cni-plugins/0.log" Jul 28 00:32:04.636005 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.635977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/egress-router-binary-copy/0.log" Jul 28 00:32:04.654837 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.654814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/cni-plugins/0.log" Jul 28 00:32:04.673809 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.673790 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/bond-cni-plugin/0.log" Jul 28 00:32:04.693724 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.693706 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/routeoverride-cni/0.log" Jul 28 00:32:04.712563 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.712539 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/whereabouts-cni-bincopy/0.log" Jul 28 00:32:04.731658 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.731641 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzh5j_3b755624-4b79-4d54-bfc2-6ac2fd632c5f/whereabouts-cni/0.log" Jul 28 00:32:04.977150 ip-10-0-128-69 kubenswrapper[2574]: I0728 00:32:04.977119 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kvhtj_66c4f215-ea76-4982-a115-bc31e5efcd77/kube-multus/0.log"