Jul 18 16:36:33.486240 ip-10-0-139-1 systemd[1]: Starting Kubernetes Kubelet... Jul 18 16:36:33.832626 ip-10-0-139-1 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 16:36:33.832626 ip-10-0-139-1 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 16:36:33.832626 ip-10-0-139-1 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 16:36:33.832626 ip-10-0-139-1 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 16:36:33.832626 ip-10-0-139-1 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 16:36:33.833452 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.832697 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 16:36:33.837515 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837492 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.837515 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837513 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.837515 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837518 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.837515 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837522 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837525 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837529 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837532 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837535 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837537 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837540 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837543 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837546 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837549 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837551 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837554 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837557 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837559 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837562 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837565 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837567 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837570 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837573 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837575 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.837699 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837578 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837581 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837583 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837586 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837588 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837591 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837597 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837599 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837602 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837604 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837607 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837609 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837612 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837615 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837619 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837622 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837625 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837628 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837630 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837633 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.838184 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837635 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837638 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837640 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837643 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837647 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837649 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837652 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837654 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837657 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837659 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837662 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837667 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837671 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837674 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837677 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837680 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837683 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837686 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837688 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.838662 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837691 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837693 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837696 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837698 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837701 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837704 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837708 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837711 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837713 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837716 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837719 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837721 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837724 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837726 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837729 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837732 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837734 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837737 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837740 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837743 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.839154 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837745 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837748 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837750 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837753 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837755 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837757 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837760 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837762 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837765 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837767 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837769 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837772 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837774 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837777 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837779 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837782 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.837784 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838499 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838507 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838510 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.839618 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838513 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838516 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838519 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838521 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838524 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838526 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838529 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838532 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838534 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838537 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838539 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838545 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838549 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838552 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838555 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838558 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838560 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838563 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838566 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.840111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838569 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838572 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838575 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838577 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838580 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838583 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838585 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838588 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838590 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838593 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838595 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838598 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838601 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838604 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838606 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838609 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838612 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838614 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838617 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838619 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.840576 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838622 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838627 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838630 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838633 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838636 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838640 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838643 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838646 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838649 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838651 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838654 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838656 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838659 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838662 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838664 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838667 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838670 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838673 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838676 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.841091 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838678 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838681 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838683 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838686 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838689 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838691 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838694 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838697 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838699 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838702 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838705 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838707 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838709 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838712 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838715 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838718 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838721 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838723 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838725 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.841545 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838730 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838733 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838736 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838739 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838741 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838743 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838746 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838749 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838752 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838754 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838757 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838759 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838762 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838764 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838767 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838769 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838771 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838774 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.838777 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.842018 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839231 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839238 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839242 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839246 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839249 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839252 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839254 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839257 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839259 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839262 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839264 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839267 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839273 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839276 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839279 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839282 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839284 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839287 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839290 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.842469 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839292 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839295 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839298 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839300 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839303 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839305 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839308 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839310 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839313 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839315 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839318 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839321 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839323 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839326 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839329 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839331 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839334 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839337 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839340 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839343 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.842971 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839345 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839348 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839350 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839353 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839357 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839362 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839365 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839368 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839371 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839373 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839376 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839379 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839382 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839384 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839387 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839390 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839393 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839395 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839398 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.843449 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839400 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839403 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839405 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839408 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839410 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839412 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839415 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839417 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839420 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839423 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839427 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839430 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839432 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839435 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839437 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839439 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839443 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839445 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839449 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839452 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.843953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839454 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839457 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839460 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839463 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839465 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839467 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839470 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839474 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839476 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839479 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839483 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839486 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839489 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839491 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839494 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839497 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839499 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839502 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839504 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839507 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.844440 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.839509 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839579 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839587 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839594 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839598 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839602 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839622 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839631 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839636 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839639 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839648 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839652 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839655 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839659 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839663 2567 flags.go:64] FLAG: --cgroup-root="" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839666 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839669 2567 flags.go:64] FLAG: --client-ca-file="" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839672 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839675 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839680 2567 flags.go:64] FLAG: --cluster-domain="" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839683 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839686 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839690 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839693 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 16:36:33.845058 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839697 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839700 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839702 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839706 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839709 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839712 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839715 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839718 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839721 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839725 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839728 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839732 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839734 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839737 2567 flags.go:64] FLAG: --enable-server="true" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839740 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839746 2567 flags.go:64] FLAG: --event-burst="100" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839749 2567 flags.go:64] FLAG: --event-qps="50" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839751 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839756 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839759 2567 flags.go:64] FLAG: --eviction-hard="" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839763 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839765 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839769 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839772 2567 flags.go:64] FLAG: --eviction-soft="" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839775 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 16:36:33.845673 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839778 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839781 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839787 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839789 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839792 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839795 2567 flags.go:64] FLAG: --feature-gates="" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839799 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839802 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839805 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839808 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839811 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839815 2567 flags.go:64] FLAG: --help="false" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839817 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-139-1.ec2.internal" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839821 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839824 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839826 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839831 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839834 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839837 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839840 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839843 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839845 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839848 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839851 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 16:36:33.846297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839854 2567 flags.go:64] FLAG: --kube-reserved="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839858 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839861 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839864 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839867 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839870 2567 flags.go:64] FLAG: --lock-file="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839873 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839882 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839885 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839891 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839895 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839898 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839901 2567 flags.go:64] FLAG: --logging-format="text" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839904 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839907 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839910 2567 flags.go:64] FLAG: --manifest-url="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839913 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839918 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839921 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839938 2567 flags.go:64] FLAG: --max-pods="110" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839942 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839945 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839947 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839950 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839953 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 16:36:33.846887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839956 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839959 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839966 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839969 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839972 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839975 2567 flags.go:64] FLAG: --pod-cidr="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839978 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839983 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839988 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839991 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839994 2567 flags.go:64] FLAG: --port="10250" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.839997 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840000 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09aead94f62513b0a" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840003 2567 flags.go:64] FLAG: --qos-reserved="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840006 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840015 2567 flags.go:64] FLAG: --register-node="true" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840018 2567 flags.go:64] FLAG: --register-with-taints="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840022 2567 flags.go:64] FLAG: --registry-burst="10" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840026 2567 flags.go:64] FLAG: --registry-qps="5" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840029 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840032 2567 flags.go:64] FLAG: --reserved-memory="" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840035 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840038 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840041 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840044 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 16:36:33.847505 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840047 2567 flags.go:64] FLAG: --runonce="false" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840050 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840053 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840055 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840058 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840061 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840064 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840067 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840070 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840073 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840076 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840079 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840082 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840084 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840087 2567 flags.go:64] FLAG: --system-cgroups="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840091 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840097 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840099 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840102 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840106 2567 flags.go:64] FLAG: --tls-min-version="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840109 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840111 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840114 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840119 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840122 2567 flags.go:64] FLAG: --v="2" Jul 18 16:36:33.848122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840128 2567 flags.go:64] FLAG: --version="false" Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840132 2567 flags.go:64] FLAG: --vmodule="" Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840136 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840139 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840270 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840274 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840277 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840280 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840283 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840286 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840289 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840291 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840294 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840297 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840299 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840302 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840305 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840307 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840310 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840312 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.848739 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840315 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840317 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840320 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840323 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840326 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840328 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840331 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840334 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840336 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840339 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840341 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840344 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840348 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840351 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840354 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840356 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840359 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840361 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840364 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840366 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.849297 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840369 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840371 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840374 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840376 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840379 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840381 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840384 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840386 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840389 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840392 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840395 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840397 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840400 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840403 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840405 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840407 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840410 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840413 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840415 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840417 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.849774 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840421 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840423 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840425 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840428 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840432 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840435 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840437 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840440 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840442 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840445 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840448 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840450 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840453 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840456 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840458 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840461 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840463 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840466 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840469 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840471 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.850267 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840474 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840476 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840479 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840483 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840486 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840489 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840492 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840495 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840499 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840502 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840504 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840507 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840509 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840512 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840514 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840517 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840520 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840526 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.850747 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840528 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840531 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840533 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840537 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.840541 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.840550 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.847361 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.847377 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.847397 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.847403 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847460 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847467 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847472 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847475 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.851204 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847478 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847482 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847485 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847487 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847490 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847493 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847495 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847498 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847500 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847503 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847506 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847509 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847512 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847515 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847518 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847520 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847523 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847526 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847528 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.851567 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847531 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847533 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847536 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847538 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847541 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847543 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847548 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847551 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847553 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847556 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847559 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847562 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847564 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847566 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847569 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847572 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847575 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847577 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847580 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847582 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.852100 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847585 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847587 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847590 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847592 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847595 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847598 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847600 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847603 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847605 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847608 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847610 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847613 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847615 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847618 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847620 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847623 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847627 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847630 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847634 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847638 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.852572 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847640 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847643 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847646 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847648 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847651 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847653 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847656 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847659 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847661 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847664 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847667 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847669 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847671 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847674 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847676 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847679 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847681 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847684 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847686 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847689 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.853086 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847691 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847693 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847697 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847700 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847703 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847705 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847708 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847710 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847713 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847715 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847718 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847720 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847723 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847726 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847729 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.853560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847731 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.847737 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847849 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847854 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847857 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847859 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847862 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847865 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847868 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847870 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847873 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847876 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847878 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847881 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847883 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 16:36:33.853911 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847886 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847888 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847891 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847894 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847896 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847898 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847901 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847903 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847906 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847908 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847911 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847913 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847916 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847919 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847922 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847940 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847944 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847947 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847950 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847953 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 16:36:33.854295 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847955 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847958 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847960 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847963 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847966 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847969 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847971 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847974 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847976 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847979 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847981 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847984 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847986 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847989 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847992 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847994 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847997 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.847999 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848001 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848004 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 16:36:33.854943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848006 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848009 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848012 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848014 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848017 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848020 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848022 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848025 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848028 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848031 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848034 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848038 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848041 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848044 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848047 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848049 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848052 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848054 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848057 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848059 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 16:36:33.855749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848061 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848064 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848066 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848069 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848071 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848074 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848077 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848080 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848084 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848087 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848090 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848093 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848095 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848098 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848102 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848105 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848108 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848111 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848114 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 16:36:33.856414 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848117 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848120 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848124 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848127 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848130 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848133 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:33.848136 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.848141 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.848256 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.850371 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.851638 2567 server.go:1019] "Starting client certificate rotation" Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.851738 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.852390 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 16:36:33.856880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.853909 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 16:36:33.860369 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.860349 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 16:36:33.868222 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.868206 2567 log.go:25] "Validated CRI v1 runtime API" Jul 18 16:36:33.870359 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.870343 2567 log.go:25] "Validated CRI v1 image API" Jul 18 16:36:33.871088 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.871071 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 16:36:33.873420 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.873400 2567 fs.go:135] Filesystem UUIDs: map[03ba2509-922d-4fc7-bdeb-75b9b7b28ac6:/dev/nvme0n1p4 0e3b452c-f6b4-4cc2-b94f-d1d115788201:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 18 16:36:33.873478 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.873419 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 16:36:33.878410 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.878300 2567 manager.go:216] Machine: {Timestamp:2026-07-18 16:36:33.877045565 +0000 UTC m=+0.288389335 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098791 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20acebaaa0e5a2754a849a2fa05891 SystemUUID:ec20aceb-aaa0-e5a2-754a-849a2fa05891 BootID:1f35d02b-b5b9-433d-be59-a78b0d169f1a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8a:0c:eb:1c:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8a:0c:eb:1c:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:fe:49:a2:91:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 16:36:33.878410 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.878405 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 16:36:33.878517 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.878484 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 16:36:33.881071 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.881051 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 16:36:33.881361 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.881332 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 16:36:33.881529 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.881367 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 16:36:33.882106 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.882094 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 16:36:33.882159 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.882111 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 16:36:33.882159 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.882121 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 16:36:33.882783 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.882772 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 16:36:33.884227 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.884216 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 16:36:33.888271 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888257 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 18 16:36:33.888414 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888402 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 16:36:33.888469 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888443 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 18 16:36:33.888469 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888458 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 16:36:33.888557 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888479 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 16:36:33.888557 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888494 2567 kubelet.go:398] "Adding apiserver pod source" Jul 18 16:36:33.888557 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888508 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 16:36:33.888955 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888920 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 16:36:33.889034 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.888965 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 16:36:33.890991 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.890966 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 16:36:33.894137 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.894119 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 16:36:33.894234 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.894159 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 16:36:33.896032 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896017 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896039 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896049 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896058 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896070 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896080 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896092 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896101 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 16:36:33.896114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896111 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 16:36:33.896348 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896121 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 16:36:33.896348 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896135 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 16:36:33.896348 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896144 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 16:36:33.896768 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896756 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 16:36:33.896822 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.896771 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 16:36:33.899771 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.899751 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s94w4" Jul 18 16:36:33.899874 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.899793 2567 server.go:1268] "Started kubelet" Jul 18 16:36:33.899910 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.899881 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 16:36:33.900034 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.899985 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 16:36:33.900136 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.900053 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 16:36:33.901122 ip-10-0-139-1 systemd[1]: Started Kubernetes Kubelet. Jul 18 16:36:33.901766 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.901749 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 16:36:33.902040 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.901861 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 16:36:33.902040 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.901949 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 16:36:33.902040 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.901944 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 16:36:33.902247 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.902234 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 18 16:36:33.905572 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.905552 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 16:36:33.906879 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.906865 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s94w4" Jul 18 16:36:33.909589 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.908756 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-1.ec2.internal.18c36f9c2b3bb1fe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-1.ec2.internal,UID:ip-10-0-139-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-1.ec2.internal,},FirstTimestamp:2026-07-18 16:36:33.899762174 +0000 UTC m=+0.311105944,LastTimestamp:2026-07-18 16:36:33.899762174 +0000 UTC m=+0.311105944,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-1.ec2.internal,}" Jul 18 16:36:33.910074 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.910060 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 16:36:33.910074 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.910067 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 16:36:33.910170 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.910155 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:33.910808 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.910791 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 16:36:33.910914 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.910811 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 16:36:33.911018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.910791 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 16:36:33.911120 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911108 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 18 16:36:33.911120 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911120 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 18 16:36:33.911649 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911632 2567 factory.go:153] Registering CRI-O factory Jul 18 16:36:33.911723 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911655 2567 factory.go:223] Registration of the crio container factory successfully Jul 18 16:36:33.911765 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911730 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 16:36:33.911765 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911743 2567 factory.go:55] Registering systemd factory Jul 18 16:36:33.911765 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911753 2567 factory.go:223] Registration of the systemd container factory successfully Jul 18 16:36:33.911909 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911782 2567 factory.go:103] Registering Raw factory Jul 18 16:36:33.911909 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.911800 2567 manager.go:1196] Started watching for new ooms in manager Jul 18 16:36:33.912445 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.912430 2567 manager.go:349] Starting recovery of all containers Jul 18 16:36:33.913238 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.913214 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 16:36:33.913344 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.913323 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 16:36:33.926099 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.926074 2567 manager.go:354] Recovery completed Jul 18 16:36:33.930180 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.930166 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:33.932479 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.932461 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:33.932567 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.932491 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:33.932567 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.932505 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:33.933068 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933055 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 16:36:33.933068 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933068 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 16:36:33.933148 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933084 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 18 16:36:33.933148 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933125 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:33.933808 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933792 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:33.933883 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933823 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:33.933883 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.933837 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:33.935202 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.935191 2567 policy_none.go:49] "None policy: Start" Jul 18 16:36:33.935265 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.935206 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 16:36:33.935265 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.935216 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 16:36:33.937223 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.937211 2567 policy_none.go:47] "Start" Jul 18 16:36:33.976778 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.976572 2567 manager.go:337] "Starting Device Plugin manager" Jul 18 16:36:33.976910 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.976802 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 16:36:33.976910 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.976819 2567 server.go:87] "Starting device plugin registration server" Jul 18 16:36:33.977070 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.977059 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 16:36:33.977102 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.977071 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 16:36:33.977205 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.977184 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 16:36:33.977322 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.977281 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 16:36:33.977322 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.977290 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 16:36:33.977854 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.977837 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 16:36:33.977942 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:33.977879 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:33.999994 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:33.999966 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 16:36:34.006586 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.001147 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 16:36:34.006586 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.001179 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 16:36:34.006586 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.001201 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 16:36:34.006586 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:34.001231 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 16:36:34.006586 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.003243 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:34.077423 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.077376 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:34.078530 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.078512 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:34.078616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.078551 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:34.078616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.078562 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:34.078616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.078585 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.086055 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.086008 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.086055 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:34.086032 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-1.ec2.internal\": node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:34.101730 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.101703 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal"] Jul 18 16:36:34.101841 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.101770 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:34.101841 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:34.101837 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:34.102685 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.102660 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:34.102765 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.102696 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:34.102765 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.102714 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:34.105162 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105149 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:34.105268 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.105363 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105289 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:34.105906 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105893 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:34.105906 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105901 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:34.106017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105920 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:34.106017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105922 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:34.106017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105969 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:34.106017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.105948 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:34.108308 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.108295 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.108364 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.108321 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 16:36:34.109543 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.109531 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientMemory" Jul 18 16:36:34.109616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.109555 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 16:36:34.109616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.109564 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeHasSufficientPID" Jul 18 16:36:34.112513 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.112497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.112597 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.112520 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.112597 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:34.112537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4359bec0e352431350db386bdadf6398-config\") pod \"kube-apiserver-proxy-ip-10-0-139-1.ec2.internal\" (UID: \"4359bec0e352431350db386bdadf6398\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.125511 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:34.125486 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-1.ec2.internal\" not found" node="ip-10-0-139-1.ec2.internal" Jul 18 16:36:34.130162 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:34.130147 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-1.ec2.internal\" not found" node="ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.475139 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.475107 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:35.485472 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.485559 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.485559 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4359bec0e352431350db386bdadf6398-config\") pod \"kube-apiserver-proxy-ip-10-0-139-1.ec2.internal\" (UID: \"4359bec0e352431350db386bdadf6398\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.485619 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485554 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.485692 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485622 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4359bec0e352431350db386bdadf6398-config\") pod \"kube-apiserver-proxy-ip-10-0-139-1.ec2.internal\" (UID: \"4359bec0e352431350db386bdadf6398\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.485692 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.485660 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a315493ff7ec57552d98ccbaa050d33e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal\" (UID: \"a315493ff7ec57552d98ccbaa050d33e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.576206 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.576164 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:35.677072 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.676999 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:35.701556 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.701530 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.706769 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:35.706649 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:35.777806 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.777763 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:35.878295 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.878264 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:35.978937 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:35.978838 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:36.017732 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.017705 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:36.079583 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:36.079554 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:36.124104 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.124072 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 16:36:36.124640 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.124252 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 16:36:36.124640 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.124293 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 16:36:36.180256 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:36.180228 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:36.184675 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.184632 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 16:31:35 +0000 UTC" deadline="2028-05-01 13:09:51.485338222 +0000 UTC" Jul 18 16:36:36.184675 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.184672 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15668h33m15.300670051s" Jul 18 16:36:36.238559 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.238528 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:36.240678 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:36.240643 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda315493ff7ec57552d98ccbaa050d33e.slice/crio-0f550a44a8f52aa1007e49860bd59c840c321dcdf80deb3dac5aa1c3c39ba30d WatchSource:0}: Error finding container 0f550a44a8f52aa1007e49860bd59c840c321dcdf80deb3dac5aa1c3c39ba30d: Status 404 returned error can't find the container with id 0f550a44a8f52aa1007e49860bd59c840c321dcdf80deb3dac5aa1c3c39ba30d Jul 18 16:36:36.240850 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:36.240832 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4359bec0e352431350db386bdadf6398.slice/crio-0fbef4f68b976c41a97f1b4fd9ed8cbc34da6e94c8317d7bd29a0be01bb22f04 WatchSource:0}: Error finding container 0fbef4f68b976c41a97f1b4fd9ed8cbc34da6e94c8317d7bd29a0be01bb22f04: Status 404 returned error can't find the container with id 0fbef4f68b976c41a97f1b4fd9ed8cbc34da6e94c8317d7bd29a0be01bb22f04 Jul 18 16:36:36.244756 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.244742 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 16:36:36.277837 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.277780 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" event={"ID":"4359bec0e352431350db386bdadf6398","Type":"ContainerStarted","Data":"0fbef4f68b976c41a97f1b4fd9ed8cbc34da6e94c8317d7bd29a0be01bb22f04"} Jul 18 16:36:36.278845 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.278821 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" event={"ID":"a315493ff7ec57552d98ccbaa050d33e","Type":"ContainerStarted","Data":"0f550a44a8f52aa1007e49860bd59c840c321dcdf80deb3dac5aa1c3c39ba30d"} Jul 18 16:36:36.280947 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:36.280932 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:36.381585 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:36.381551 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-1.ec2.internal\" not found" Jul 18 16:36:36.422353 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.422326 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:36.483052 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.482973 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" Jul 18 16:36:36.493613 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.493593 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 16:36:36.494554 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.494541 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" Jul 18 16:36:36.502378 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:36.502361 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 16:36:37.162398 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.162359 2567 apiserver.go:52] "Watching apiserver" Jul 18 16:36:37.172690 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.172665 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 16:36:37.173114 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.173079 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-4s8cp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal","openshift-multus/multus-additional-cni-plugins-hswxz","openshift-multus/multus-hqwj6","openshift-ovn-kubernetes/ovnkube-node-smlc5","kube-system/global-pull-secret-syncer-sc698","kube-system/konnectivity-agent-wq4kr","kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal","openshift-dns/node-resolver-tz5zg","openshift-multus/network-metrics-daemon-gzzf2","openshift-network-diagnostics/network-check-target-dqnmp","openshift-network-operator/iptables-alerter-mtjsx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv","openshift-cluster-node-tuning-operator/tuned-24g48"] Jul 18 16:36:37.175469 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.175448 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.178029 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.178009 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 16:36:37.178137 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.178092 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 16:36:37.178194 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.178133 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-twmsc\"" Jul 18 16:36:37.178337 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.178317 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.180362 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.180342 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.180552 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.180490 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 16:36:37.180708 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.180687 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.180817 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.180736 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.180817 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.180743 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n9z45\"" Jul 18 16:36:37.182846 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.182814 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183242 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183259 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183369 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183463 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183605 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q7xvg\"" Jul 18 16:36:37.183705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183613 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 16:36:37.183986 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.183757 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.186845 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.186674 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 16:36:37.186845 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.186818 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j7zrw\"" Jul 18 16:36:37.187025 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.186928 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:37.187801 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.187107 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.189697 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.189680 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ds79f\"" Jul 18 16:36:37.189911 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.189892 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.190001 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.189962 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:37.190432 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190412 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.190542 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190456 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 16:36:37.190542 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190463 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 16:36:37.190542 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190487 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 16:36:37.190542 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190532 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 16:36:37.190793 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.190666 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.192100 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.192080 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.194170 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.194140 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-25lps\"" Jul 18 16:36:37.194260 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.194145 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.194317 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.194298 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.194435 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.194421 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.195218 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-multus\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.195306 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195231 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-kubelet\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195306 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/278663ef-6a36-4fde-9a07-bd028a1e19c3-konnectivity-ca\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.195306 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195296 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-system-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.195446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195322 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-systemd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-ovn\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovn-node-metrics-cert\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195392 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k9fc\" (UniqueName: \"kubernetes.io/projected/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-kube-api-access-6k9fc\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-sys-fs\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195455 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhcng\" (UniqueName: \"kubernetes.io/projected/af604576-50db-432f-9e31-bbf5d449a913-kube-api-access-zhcng\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzv89\" (UniqueName: \"kubernetes.io/projected/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kube-api-access-dzv89\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195562 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-os-release\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195584 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-script-lib\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-binary-copy\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.195698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-conf-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195707 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-socket-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195735 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-registration-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-system-cni-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195811 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-k8s-cni-cncf-io\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195835 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-config\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-cnibin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195898 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195926 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwfqm\" (UniqueName: \"kubernetes.io/projected/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-kube-api-access-pwfqm\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-slash\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.195965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/278663ef-6a36-4fde-9a07-bd028a1e19c3-agent-certs\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.196018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196002 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196037 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-bin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196061 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-hostroot\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-device-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196144 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-os-release\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-multus-certs\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-etc-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-log-socket\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196267 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-env-overrides\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196347 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cnibin\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-cni-binary-copy\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-etc-kubernetes\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-var-lib-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.196497 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196519 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196553 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-multus-daemon-config\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196573 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-bin\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196650 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-systemd-units\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196676 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-netns\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196687 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-th6cn\"" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-node-log\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-netd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196744 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196745 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196801 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196737 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196904 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-netns\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196927 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-kubelet\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-socket-dir-parent\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.196999 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.197105 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.197046 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.199175 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.199143 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.199267 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.199160 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.199267 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.199184 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z2lp5\"" Jul 18 16:36:37.199598 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.199400 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.199598 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.199482 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:37.202445 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.202429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:37.202558 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.202480 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:37.204919 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.204885 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.208015 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.207895 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 16:36:37.208015 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.207922 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 16:36:37.208015 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.207924 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 16:36:37.208225 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.208054 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mdp7k\"" Jul 18 16:36:37.215618 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.215600 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lxjcl" Jul 18 16:36:37.220615 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.220595 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lxjcl" Jul 18 16:36:37.284684 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.284658 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 16:36:37.297999 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.297951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-socket-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.297999 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.297985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-registration-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298014 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-system-cni-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-k8s-cni-cncf-io\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298081 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-config\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298111 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-registration-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-lib-modules\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298176 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-socket-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.298207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-cnibin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-run\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298260 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-sys\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298287 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwfqm\" (UniqueName: \"kubernetes.io/projected/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-kube-api-access-pwfqm\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298287 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298238 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-k8s-cni-cncf-io\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298314 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-system-cni-dir\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298341 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-slash\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-cnibin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298397 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f30a709d-f80b-4df8-8578-e3a6adce6768-host-slash\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-kubelet-config\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298397 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-slash\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/278663ef-6a36-4fde-9a07-bd028a1e19c3-agent-certs\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298502 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298633 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-bin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.298683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298663 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-hostroot\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68jbp\" (UniqueName: \"kubernetes.io/projected/f30a709d-f80b-4df8-8578-e3a6adce6768-kube-api-access-68jbp\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298717 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-device-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298731 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-os-release\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298746 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-multus-certs\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298762 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-etc-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298776 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-log-socket\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-env-overrides\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298832 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298859 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-device-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298903 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-dbus\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cnibin\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-cni-binary-copy\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.298987 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-etc-kubernetes\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299047 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-config\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299043 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkmsx\" (UniqueName: \"kubernetes.io/projected/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-kube-api-access-lkmsx\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.299441 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299065 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299086 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-env-overrides\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299092 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5379291e-07ab-4173-83da-9ee82abd6e42-host\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299120 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-etc-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299121 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-multus-certs\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299123 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-var-lib-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-bin\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299165 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-hostroot\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299165 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-log-socket\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299187 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-var-lib-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299215 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-etc-kubernetes\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299225 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cnibin\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299258 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299263 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-openvswitch\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299270 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-os-release\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-ovn-kubernetes\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.300264 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-modprobe-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299334 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-conf\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299367 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-multus-daemon-config\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-bin\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-host\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-systemd-units\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299602 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-bin\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-netns\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299652 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-systemd-units\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-node-log\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299685 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-cni-binary-copy\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-run-netns\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-node-log\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-netd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299768 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-cni-netd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299805 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5379291e-07ab-4173-83da-9ee82abd6e42-serviceca\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.301067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299831 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299848 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af604576-50db-432f-9e31-bbf5d449a913-multus-daemon-config\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-netns\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299910 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-kubelet\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299970 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-kubelet\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299970 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/940fc19a-b412-455e-ba83-285f02e11730-tmp-dir\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299944 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-run-netns\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.299996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-var-lib-kubelet\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300016 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhdb4\" (UniqueName: \"kubernetes.io/projected/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-kube-api-access-hhdb4\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-socket-dir-parent\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300051 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300069 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-socket-dir-parent\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300075 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-multus\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-kubelet\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300122 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr9lj\" (UniqueName: \"kubernetes.io/projected/5379291e-07ab-4173-83da-9ee82abd6e42-kube-api-access-xr9lj\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300135 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-host-var-lib-cni-multus\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300146 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-host-kubelet\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-systemd\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300209 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/278663ef-6a36-4fde-9a07-bd028a1e19c3-konnectivity-ca\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300234 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-system-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-systemd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300277 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-ovn\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300289 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-system-cni-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300301 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovn-node-metrics-cert\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k9fc\" (UniqueName: \"kubernetes.io/projected/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-kube-api-access-6k9fc\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300348 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysconfig\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-tmp\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-sys-fs\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhcng\" (UniqueName: \"kubernetes.io/projected/af604576-50db-432f-9e31-bbf5d449a913-kube-api-access-zhcng\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.302885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f30a709d-f80b-4df8-8578-e3a6adce6768-iptables-alerter-script\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzv89\" (UniqueName: \"kubernetes.io/projected/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kube-api-access-dzv89\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-os-release\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300492 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-script-lib\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qr6\" (UniqueName: \"kubernetes.io/projected/940fc19a-b412-455e-ba83-285f02e11730-kube-api-access-j7qr6\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-binary-copy\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300595 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-conf-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300607 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/278663ef-6a36-4fde-9a07-bd028a1e19c3-konnectivity-ca\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/940fc19a-b412-455e-ba83-285f02e11730-hosts-file\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300672 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-multus-conf-dir\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300670 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-kubernetes\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300673 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300714 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-ovn\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-run-systemd\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300754 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-tuned\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300844 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-sys-fs\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.303525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.300942 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af604576-50db-432f-9e31-bbf5d449a913-os-release\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.304166 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.301176 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-cni-binary-copy\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.304166 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.301303 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovnkube-script-lib\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.304166 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.302437 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/278663ef-6a36-4fde-9a07-bd028a1e19c3-agent-certs\") pod \"konnectivity-agent-wq4kr\" (UID: \"278663ef-6a36-4fde-9a07-bd028a1e19c3\") " pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.304166 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.303229 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-ovn-node-metrics-cert\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.305584 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.305560 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwfqm\" (UniqueName: \"kubernetes.io/projected/d9291fc5-ce9c-4fec-a60a-27ec7a2ba957-kube-api-access-pwfqm\") pod \"multus-additional-cni-plugins-hswxz\" (UID: \"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957\") " pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.311011 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.310988 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhcng\" (UniqueName: \"kubernetes.io/projected/af604576-50db-432f-9e31-bbf5d449a913-kube-api-access-zhcng\") pod \"multus-hqwj6\" (UID: \"af604576-50db-432f-9e31-bbf5d449a913\") " pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.311819 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.311793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k9fc\" (UniqueName: \"kubernetes.io/projected/72d5a44e-acb8-4ab5-99d3-ddd9c36704df-kube-api-access-6k9fc\") pod \"ovnkube-node-smlc5\" (UID: \"72d5a44e-acb8-4ab5-99d3-ddd9c36704df\") " pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.312190 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.312173 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzv89\" (UniqueName: \"kubernetes.io/projected/ad6a4e22-a5f1-4c8c-843a-83f1ce511390-kube-api-access-dzv89\") pod \"aws-ebs-csi-driver-node-cwcvv\" (UID: \"ad6a4e22-a5f1-4c8c-843a-83f1ce511390\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.401375 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-run\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.401375 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-sys\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f30a709d-f80b-4df8-8578-e3a6adce6768-host-slash\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-kubelet-config\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401495 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-run\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jbp\" (UniqueName: \"kubernetes.io/projected/f30a709d-f80b-4df8-8578-e3a6adce6768-kube-api-access-68jbp\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401552 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f30a709d-f80b-4df8-8578-e3a6adce6768-host-slash\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-dbus\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401567 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-kubelet-config\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401503 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-sys\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.401637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401623 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkmsx\" (UniqueName: \"kubernetes.io/projected/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-kube-api-access-lkmsx\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401628 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8a69b23-4563-44fd-825b-d3a148575aec-dbus\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401651 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5379291e-07ab-4173-83da-9ee82abd6e42-host\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-modprobe-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-conf\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401747 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5379291e-07ab-4173-83da-9ee82abd6e42-host\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.401802 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-host\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.401869 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:37.901847133 +0000 UTC m=+3.040609788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401870 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-modprobe-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401866 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-conf\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401900 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-host\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5379291e-07ab-4173-83da-9ee82abd6e42-serviceca\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401942 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401969 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/940fc19a-b412-455e-ba83-285f02e11730-tmp-dir\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.401991 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-var-lib-kubelet\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhdb4\" (UniqueName: \"kubernetes.io/projected/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-kube-api-access-hhdb4\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402048 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysctl-d\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr9lj\" (UniqueName: \"kubernetes.io/projected/5379291e-07ab-4173-83da-9ee82abd6e42-kube-api-access-xr9lj\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402068 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-var-lib-kubelet\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402119 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-systemd\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysconfig\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402175 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-tmp\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402196 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f30a709d-f80b-4df8-8578-e3a6adce6768-iptables-alerter-script\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qr6\" (UniqueName: \"kubernetes.io/projected/940fc19a-b412-455e-ba83-285f02e11730-kube-api-access-j7qr6\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.402226 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402242 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/940fc19a-b412-455e-ba83-285f02e11730-hosts-file\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-kubernetes\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402217 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/940fc19a-b412-455e-ba83-285f02e11730-tmp-dir\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402263 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-sysconfig\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.402278 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:36:37.90226189 +0000 UTC m=+3.041024546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402301 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-systemd\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.402809 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-tuned\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402310 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/940fc19a-b412-455e-ba83-285f02e11730-hosts-file\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402383 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-lib-modules\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402387 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5379291e-07ab-4173-83da-9ee82abd6e42-serviceca\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402536 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-lib-modules\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402658 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-kubernetes\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.403580 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.402860 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f30a709d-f80b-4df8-8578-e3a6adce6768-iptables-alerter-script\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.404793 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.404767 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-etc-tuned\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.404919 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.404826 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-tmp\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.409245 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.409210 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:37.409245 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.409241 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:37.409471 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.409255 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:37.409471 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.409341 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:37.909322979 +0000 UTC m=+3.048085642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:37.410579 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.410531 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkmsx\" (UniqueName: \"kubernetes.io/projected/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-kube-api-access-lkmsx\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.410954 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.410929 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68jbp\" (UniqueName: \"kubernetes.io/projected/f30a709d-f80b-4df8-8578-e3a6adce6768-kube-api-access-68jbp\") pod \"iptables-alerter-mtjsx\" (UID: \"f30a709d-f80b-4df8-8578-e3a6adce6768\") " pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.411611 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.411577 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qr6\" (UniqueName: \"kubernetes.io/projected/940fc19a-b412-455e-ba83-285f02e11730-kube-api-access-j7qr6\") pod \"node-resolver-tz5zg\" (UID: \"940fc19a-b412-455e-ba83-285f02e11730\") " pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.411982 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.411961 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhdb4\" (UniqueName: \"kubernetes.io/projected/bc05ae47-b5c5-4891-9419-ac9cd50c1dcc-kube-api-access-hhdb4\") pod \"tuned-24g48\" (UID: \"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc\") " pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.412062 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.411961 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr9lj\" (UniqueName: \"kubernetes.io/projected/5379291e-07ab-4173-83da-9ee82abd6e42-kube-api-access-xr9lj\") pod \"node-ca-4s8cp\" (UID: \"5379291e-07ab-4173-83da-9ee82abd6e42\") " pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.487139 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.487046 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:37.495490 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.495470 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" Jul 18 16:36:37.504367 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.504336 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hswxz" Jul 18 16:36:37.510377 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.510357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hqwj6" Jul 18 16:36:37.517288 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.517266 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:36:37.524314 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.524295 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tz5zg" Jul 18 16:36:37.534095 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.534067 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4s8cp" Jul 18 16:36:37.539150 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.539126 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24g48" Jul 18 16:36:37.545794 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.545772 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mtjsx" Jul 18 16:36:37.780109 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.780020 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 16:36:37.906723 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.906690 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:37.906880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:37.906748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:37.906880 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.906853 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:37.906995 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.906904 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:38.906889359 +0000 UTC m=+4.045652013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:37.906995 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.906854 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:37.906995 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:37.906981 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:36:38.906968542 +0000 UTC m=+4.045731188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:38.007129 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.007089 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:38.007307 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.007239 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:38.007307 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.007264 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:38.007307 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.007276 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:38.007470 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.007335 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:39.00731838 +0000 UTC m=+4.146081040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:38.120400 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.120375 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9291fc5_ce9c_4fec_a60a_27ec7a2ba957.slice/crio-40216050e296c10fcc43158b23eba7eb9eedbe1c627f705e390c8d9bec9904e1 WatchSource:0}: Error finding container 40216050e296c10fcc43158b23eba7eb9eedbe1c627f705e390c8d9bec9904e1: Status 404 returned error can't find the container with id 40216050e296c10fcc43158b23eba7eb9eedbe1c627f705e390c8d9bec9904e1 Jul 18 16:36:38.121378 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.121356 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc05ae47_b5c5_4891_9419_ac9cd50c1dcc.slice/crio-4b714632470808cc499baace38fde360fa5b6768dadfca7c847560c53cde52de WatchSource:0}: Error finding container 4b714632470808cc499baace38fde360fa5b6768dadfca7c847560c53cde52de: Status 404 returned error can't find the container with id 4b714632470808cc499baace38fde360fa5b6768dadfca7c847560c53cde52de Jul 18 16:36:38.124111 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.123982 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278663ef_6a36_4fde_9a07_bd028a1e19c3.slice/crio-80e6bd6ed970d51958429a35895aa2d3cd187850b4babaa689d058d53e6824a2 WatchSource:0}: Error finding container 80e6bd6ed970d51958429a35895aa2d3cd187850b4babaa689d058d53e6824a2: Status 404 returned error can't find the container with id 80e6bd6ed970d51958429a35895aa2d3cd187850b4babaa689d058d53e6824a2 Jul 18 16:36:38.126722 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.126690 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6a4e22_a5f1_4c8c_843a_83f1ce511390.slice/crio-41f94856fb24f2902d355db1432ae9f65d8cbfaa239687410fd662f58c0ed125 WatchSource:0}: Error finding container 41f94856fb24f2902d355db1432ae9f65d8cbfaa239687410fd662f58c0ed125: Status 404 returned error can't find the container with id 41f94856fb24f2902d355db1432ae9f65d8cbfaa239687410fd662f58c0ed125 Jul 18 16:36:38.128153 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.128130 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf604576_50db_432f_9e31_bbf5d449a913.slice/crio-07b1a845eed778544f26917a596d201472aa8fc3a085a4768bd43c883b4a3aac WatchSource:0}: Error finding container 07b1a845eed778544f26917a596d201472aa8fc3a085a4768bd43c883b4a3aac: Status 404 returned error can't find the container with id 07b1a845eed778544f26917a596d201472aa8fc3a085a4768bd43c883b4a3aac Jul 18 16:36:38.129092 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.129064 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30a709d_f80b_4df8_8578_e3a6adce6768.slice/crio-47812884157ba95efced3821340392607df35e6407c4bfccb5ed0b697cdfa0e6 WatchSource:0}: Error finding container 47812884157ba95efced3821340392607df35e6407c4bfccb5ed0b697cdfa0e6: Status 404 returned error can't find the container with id 47812884157ba95efced3821340392607df35e6407c4bfccb5ed0b697cdfa0e6 Jul 18 16:36:38.130378 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.130342 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940fc19a_b412_455e_ba83_285f02e11730.slice/crio-511df31a46e65a10413741f87eb047ae049dc3a4c4a4801eaec1030ab78c4da5 WatchSource:0}: Error finding container 511df31a46e65a10413741f87eb047ae049dc3a4c4a4801eaec1030ab78c4da5: Status 404 returned error can't find the container with id 511df31a46e65a10413741f87eb047ae049dc3a4c4a4801eaec1030ab78c4da5 Jul 18 16:36:38.131749 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.131358 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d5a44e_acb8_4ab5_99d3_ddd9c36704df.slice/crio-485cf5a3b3bb8a9f1fcfe33936cc21e5d6a36c71aaf7d581efe4029e218a22e5 WatchSource:0}: Error finding container 485cf5a3b3bb8a9f1fcfe33936cc21e5d6a36c71aaf7d581efe4029e218a22e5: Status 404 returned error can't find the container with id 485cf5a3b3bb8a9f1fcfe33936cc21e5d6a36c71aaf7d581efe4029e218a22e5 Jul 18 16:36:38.133439 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:36:38.133241 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5379291e_07ab_4173_83da_9ee82abd6e42.slice/crio-feee489e6cc4d2b71c4d103dfd197a5873a6c610e37f8b4be7349bc1d4bb7167 WatchSource:0}: Error finding container feee489e6cc4d2b71c4d103dfd197a5873a6c610e37f8b4be7349bc1d4bb7167: Status 404 returned error can't find the container with id feee489e6cc4d2b71c4d103dfd197a5873a6c610e37f8b4be7349bc1d4bb7167 Jul 18 16:36:38.224285 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.224233 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 16:31:37 +0000 UTC" deadline="2028-01-10 18:06:39.311064096 +0000 UTC" Jul 18 16:36:38.224285 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.224278 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12985h30m1.086788072s" Jul 18 16:36:38.283846 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.283807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wq4kr" event={"ID":"278663ef-6a36-4fde-9a07-bd028a1e19c3","Type":"ContainerStarted","Data":"80e6bd6ed970d51958429a35895aa2d3cd187850b4babaa689d058d53e6824a2"} Jul 18 16:36:38.284828 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.284805 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4s8cp" event={"ID":"5379291e-07ab-4173-83da-9ee82abd6e42","Type":"ContainerStarted","Data":"feee489e6cc4d2b71c4d103dfd197a5873a6c610e37f8b4be7349bc1d4bb7167"} Jul 18 16:36:38.285867 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.285849 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mtjsx" event={"ID":"f30a709d-f80b-4df8-8578-e3a6adce6768","Type":"ContainerStarted","Data":"47812884157ba95efced3821340392607df35e6407c4bfccb5ed0b697cdfa0e6"} Jul 18 16:36:38.286956 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.286931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" event={"ID":"ad6a4e22-a5f1-4c8c-843a-83f1ce511390","Type":"ContainerStarted","Data":"41f94856fb24f2902d355db1432ae9f65d8cbfaa239687410fd662f58c0ed125"} Jul 18 16:36:38.287987 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.287968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hqwj6" event={"ID":"af604576-50db-432f-9e31-bbf5d449a913","Type":"ContainerStarted","Data":"07b1a845eed778544f26917a596d201472aa8fc3a085a4768bd43c883b4a3aac"} Jul 18 16:36:38.289037 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.289016 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24g48" event={"ID":"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc","Type":"ContainerStarted","Data":"4b714632470808cc499baace38fde360fa5b6768dadfca7c847560c53cde52de"} Jul 18 16:36:38.290164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.290140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerStarted","Data":"40216050e296c10fcc43158b23eba7eb9eedbe1c627f705e390c8d9bec9904e1"} Jul 18 16:36:38.292109 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.292087 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" event={"ID":"4359bec0e352431350db386bdadf6398","Type":"ContainerStarted","Data":"22a8e802c0a307741d9097bf619c835a582ed1bc18a588ccaa76e0b6367611ac"} Jul 18 16:36:38.293339 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.293310 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"485cf5a3b3bb8a9f1fcfe33936cc21e5d6a36c71aaf7d581efe4029e218a22e5"} Jul 18 16:36:38.294358 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.294340 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tz5zg" event={"ID":"940fc19a-b412-455e-ba83-285f02e11730","Type":"ContainerStarted","Data":"511df31a46e65a10413741f87eb047ae049dc3a4c4a4801eaec1030ab78c4da5"} Jul 18 16:36:38.306242 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.306194 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-1.ec2.internal" podStartSLOduration=2.30617933 podStartE2EDuration="2.30617933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:38.305889729 +0000 UTC m=+3.444652402" watchObservedRunningTime="2026-07-18 16:36:38.30617933 +0000 UTC m=+3.444941992" Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.913245 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:38.913323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.913455 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.913535 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:36:40.913501224 +0000 UTC m=+6.052263866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.913930 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:38.914017 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:38.913979 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:40.913965391 +0000 UTC m=+6.052728035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:39.013726 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.013685 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:39.013969 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.013949 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:39.014039 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.013977 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:39.014039 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.013992 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:39.014142 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.014053 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:41.014033569 +0000 UTC m=+6.152796230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:39.224795 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.224670 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 16:31:37 +0000 UTC" deadline="2028-02-02 08:47:23.853449462 +0000 UTC" Jul 18 16:36:39.224795 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.224709 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13528h10m44.628744703s" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.274622 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.274628 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.274761 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.274841 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.274903 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:39.275093 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:39.274981 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:39.302783 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.302746 2567 generic.go:356] "Generic (PLEG): container finished" podID="a315493ff7ec57552d98ccbaa050d33e" containerID="d398150a5840337ae93c8818b7589051f2f852cef4e662cf107e13b94e0bd3c0" exitCode=0 Jul 18 16:36:39.303361 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:39.303338 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" event={"ID":"a315493ff7ec57552d98ccbaa050d33e","Type":"ContainerDied","Data":"d398150a5840337ae93c8818b7589051f2f852cef4e662cf107e13b94e0bd3c0"} Jul 18 16:36:40.322267 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:40.321333 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" event={"ID":"a315493ff7ec57552d98ccbaa050d33e","Type":"ContainerStarted","Data":"d32cb6deaa7afac8a8203f51a17feb0b15dcb88a87aca77b537b42897ce5ecf0"} Jul 18 16:36:40.930209 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:40.929998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:40.930209 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:40.930065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:40.930209 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:40.930217 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:40.930536 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:40.930282 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:36:44.930263113 +0000 UTC m=+10.069025777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:40.930790 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:40.930767 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:40.930884 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:40.930831 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:44.930814468 +0000 UTC m=+10.069577122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:41.030721 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:41.030679 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:41.030929 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.030881 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:41.030929 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.030905 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:41.030929 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.030917 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:41.031101 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.030975 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:45.030957142 +0000 UTC m=+10.169719796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:41.275050 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:41.274919 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:41.275050 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.275040 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:41.275439 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:41.275288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:41.275439 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:41.274925 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:41.275439 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.275403 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:41.275646 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:41.275555 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:43.274387 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:43.274324 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:43.274887 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:43.274462 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:43.274887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:43.274476 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:43.274887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:43.274536 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:43.274887 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:43.274648 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:43.274887 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:43.274732 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:44.964313 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:44.964273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:44.964801 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:44.964335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:44.964801 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:44.964438 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:44.964801 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:44.964497 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:36:52.964482743 +0000 UTC m=+18.103245387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:44.964801 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:44.964438 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:44.964801 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:44.964572 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:52.964557666 +0000 UTC m=+18.103320323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:45.065223 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:45.065188 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:45.065463 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.065414 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:45.065463 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.065439 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:45.065463 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.065452 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:45.065685 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.065539 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:36:53.065501709 +0000 UTC m=+18.204264372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:45.275710 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:45.275628 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:45.275873 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.275730 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:45.275873 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:45.275857 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:45.276029 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.276003 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:45.276109 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:45.276049 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:45.276169 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:45.276148 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:47.274535 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:47.274432 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:47.275073 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:47.274592 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:47.275073 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:47.274624 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:47.275073 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:47.274660 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:47.275073 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:47.274759 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:47.275073 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:47.274844 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:49.274452 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:49.274410 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:49.274944 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:49.274427 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:49.274944 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:49.274563 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:49.274944 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:49.274427 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:49.274944 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:49.274682 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:49.274944 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:49.274789 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:51.274084 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:51.274050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:51.274551 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:51.274102 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:51.274551 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:51.274062 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:51.274551 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:51.274183 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:51.274551 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:51.274275 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:51.274551 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:51.274359 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:53.026088 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.026054 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:53.026644 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.026118 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:53.026644 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.026247 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:53.026644 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.026262 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:53.026644 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.026315 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:37:09.026294623 +0000 UTC m=+34.165057284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:36:53.026644 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.026331 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:37:09.026323753 +0000 UTC m=+34.165086394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:36:53.126762 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.126726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:53.126939 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.126886 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:36:53.126939 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.126913 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:36:53.126939 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.126927 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:53.127049 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.126990 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:37:09.126968439 +0000 UTC m=+34.265731100 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:36:53.278117 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.278023 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:53.278284 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.278140 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:53.278284 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:53.278174 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:53.278284 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.278172 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:53.278284 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.278255 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:53.278479 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:53.278339 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:55.275224 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:55.275196 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:55.275675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:55.275299 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:55.275675 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:55.275368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:55.275675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:55.275478 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:55.275675 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:55.275544 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:55.275675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:55.275621 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:57.277520 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.277278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:57.278098 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.277288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:57.278098 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:57.277649 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:57.278098 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.277288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:57.278098 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:57.277691 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:57.278098 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:57.277734 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:57.356236 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.356206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" event={"ID":"ad6a4e22-a5f1-4c8c-843a-83f1ce511390","Type":"ContainerStarted","Data":"e8e375087eb8fadd8e19958c44f5e316f8ff7502466c0e4c7b91b0412b5d53b3"} Jul 18 16:36:57.357853 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.357822 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hqwj6" event={"ID":"af604576-50db-432f-9e31-bbf5d449a913","Type":"ContainerStarted","Data":"5200917ccf33a81625678d3cc52e0b1b1b80a209616793291fa9487710c61d88"} Jul 18 16:36:57.359461 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.359433 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24g48" event={"ID":"bc05ae47-b5c5-4891-9419-ac9cd50c1dcc","Type":"ContainerStarted","Data":"2ecd78ba090fd44997f0ef5a0d8f9432fb358dcf66ba3d8de417752b822b09ef"} Jul 18 16:36:57.361158 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.361129 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="bf2dfd9be4c3283ea7f73fe45ccb652bb1db5584b208d07d2c18dde9c2653632" exitCode=0 Jul 18 16:36:57.361268 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.361202 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"bf2dfd9be4c3283ea7f73fe45ccb652bb1db5584b208d07d2c18dde9c2653632"} Jul 18 16:36:57.363473 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.363455 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:36:57.363860 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.363840 2567 generic.go:356] "Generic (PLEG): container finished" podID="72d5a44e-acb8-4ab5-99d3-ddd9c36704df" containerID="5dbf8978dd5818e2af0d9b9edd3b3f127a6bfa660baabc2e2774a794ab649a12" exitCode=1 Jul 18 16:36:57.363946 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.363903 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"f5a9c09f465bbdabc40deff1d882bdb793a79873f802f7f5f3028ffd9990f46e"} Jul 18 16:36:57.364007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.363958 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"71435302d654525f481a90b4532244f47bbdfcfa93aba01b91f02421bfea229e"} Jul 18 16:36:57.364007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.363991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerDied","Data":"5dbf8978dd5818e2af0d9b9edd3b3f127a6bfa660baabc2e2774a794ab649a12"} Jul 18 16:36:57.364103 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.364034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"b596c9575b9cc5c4d6f4206cb20bb21a6f514e3ad977b9aa3642fb69245831cb"} Jul 18 16:36:57.365235 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.365217 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tz5zg" event={"ID":"940fc19a-b412-455e-ba83-285f02e11730","Type":"ContainerStarted","Data":"8ee49ad2f877ecc9de4a52b08271ad2fe672bf22ef8daf9ff3d22638cc92e696"} Jul 18 16:36:57.366577 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.366544 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wq4kr" event={"ID":"278663ef-6a36-4fde-9a07-bd028a1e19c3","Type":"ContainerStarted","Data":"0f9d3652cafa7ee173b2edb7b1035794833bf1f7a4c46bcb6020999b89442693"} Jul 18 16:36:57.367961 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.367936 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4s8cp" event={"ID":"5379291e-07ab-4173-83da-9ee82abd6e42","Type":"ContainerStarted","Data":"45889da18e210ffd9a393a65f3d28c385e7e59c8c99fcffc0fdf8a3607b79417"} Jul 18 16:36:57.376575 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.376466 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hqwj6" podStartSLOduration=3.321807461 podStartE2EDuration="22.376433662s" totalImagesPullingTime="19.054626201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:57.376416115 +0000 UTC m=+22.515178781" watchObservedRunningTime="2026-07-18 16:36:57.376433662 +0000 UTC m=+22.515196315" Jul 18 16:36:57.377122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.377089 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-1.ec2.internal" podStartSLOduration=21.377079938 podStartE2EDuration="21.377079938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:40.342738864 +0000 UTC m=+5.481501528" watchObservedRunningTime="2026-07-18 16:36:57.377079938 +0000 UTC m=+22.515842600" Jul 18 16:36:57.391755 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.391122 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4s8cp" podStartSLOduration=3.643361825 podStartE2EDuration="22.391101659s" totalImagesPullingTime="18.747739834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:57.390463189 +0000 UTC m=+22.529225852" watchObservedRunningTime="2026-07-18 16:36:57.391101659 +0000 UTC m=+22.529864322" Jul 18 16:36:57.432972 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.432925 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tz5zg" podStartSLOduration=3.978655893 podStartE2EDuration="22.432908225s" totalImagesPullingTime="18.454252332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:57.432856332 +0000 UTC m=+22.571619013" watchObservedRunningTime="2026-07-18 16:36:57.432908225 +0000 UTC m=+22.571670880" Jul 18 16:36:57.451433 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.451383 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-24g48" podStartSLOduration=3.693223785 podStartE2EDuration="22.451362109s" totalImagesPullingTime="18.758138324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:57.451203696 +0000 UTC m=+22.589966358" watchObservedRunningTime="2026-07-18 16:36:57.451362109 +0000 UTC m=+22.590124773" Jul 18 16:36:57.466640 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.466602 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wq4kr" podStartSLOduration=8.164799423 podStartE2EDuration="22.466588408s" totalImagesPullingTime="14.301788985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:57.466309303 +0000 UTC m=+22.605071967" watchObservedRunningTime="2026-07-18 16:36:57.466588408 +0000 UTC m=+22.605351070" Jul 18 16:36:57.547665 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.547626 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:57.548253 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:57.548233 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:58.355042 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.355018 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 16:36:58.373746 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.373726 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:36:58.374208 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.374185 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"261722884e3f6cfe72cac7b6bc08cb9ccb8df68ea690e2b40d7e566411fbd604"} Jul 18 16:36:58.374301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.374269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"5bef430f6babac887eaa838161143cd3c3d71499053b0c87da0161403c825f34"} Jul 18 16:36:58.376138 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.376113 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" event={"ID":"ad6a4e22-a5f1-4c8c-843a-83f1ce511390","Type":"ContainerStarted","Data":"c016608e3a3b221c34ee81d6c3e2f64ccf2bd3fd9b5a3dc2d857126f29bd303e"} Jul 18 16:36:58.376875 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.376846 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:58.377393 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:58.377378 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wq4kr" Jul 18 16:36:59.262952 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.262850 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T16:36:58.355036987Z","UUID":"59e840d1-5346-4545-adad-b3356d6be3e7","Handler":null,"Name":"","Endpoint":""} Jul 18 16:36:59.266049 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.266019 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 16:36:59.266191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.266066 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 16:36:59.278599 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.278568 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:36:59.278599 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.278595 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:36:59.278793 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.278601 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:36:59.278793 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:59.278677 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:36:59.278793 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:59.278753 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:36:59.278936 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:36:59.278823 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:36:59.379768 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.379732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mtjsx" event={"ID":"f30a709d-f80b-4df8-8578-e3a6adce6768","Type":"ContainerStarted","Data":"71651167eb05035904d3c4085b31f46efe83a718f6bcb8833b1a8a9487ff4625"} Jul 18 16:36:59.394659 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:36:59.394614 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mtjsx" podStartSLOduration=5.643085768 podStartE2EDuration="24.394600521s" totalImagesPullingTime="18.751514753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:36:59.394394771 +0000 UTC m=+24.533157433" watchObservedRunningTime="2026-07-18 16:36:59.394600521 +0000 UTC m=+24.533363206" Jul 18 16:37:00.385618 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:00.385402 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" event={"ID":"ad6a4e22-a5f1-4c8c-843a-83f1ce511390","Type":"ContainerStarted","Data":"09cb3880d40a9b6051bb385ba9f6dc485abe04e40075610075161069261d234b"} Jul 18 16:37:00.388326 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:00.388304 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:37:00.388638 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:00.388619 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"275a85d970734d4cdcdf1ca5b481594c5d18058e12819b5d0cf7459b86253905"} Jul 18 16:37:00.402488 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:00.402448 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwcvv" podStartSLOduration=4.387483212 podStartE2EDuration="25.402435921s" totalImagesPullingTime="21.014952709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:00.401907414 +0000 UTC m=+25.540670092" watchObservedRunningTime="2026-07-18 16:37:00.402435921 +0000 UTC m=+25.541198584" Jul 18 16:37:01.274330 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:01.274289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:01.274575 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:01.274289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:01.274575 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:01.274426 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:01.274575 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:01.274299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:01.274575 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:01.274500 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:37:01.274795 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:01.274609 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:02.395890 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.395668 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerStarted","Data":"9a1be9bb926aa5a071fa48f98be0fc45540e7856753f80e31b78e8bafb1189b7"} Jul 18 16:37:02.400358 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.400332 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:37:02.401246 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.401219 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"0b63395f526d91781549651867768fdb461e7ed45803301830078913b1db223a"} Jul 18 16:37:02.401527 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.401492 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:02.401624 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.401573 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:02.401624 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.401623 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:02.401779 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.401736 2567 scope.go:117] "RemoveContainer" containerID="5dbf8978dd5818e2af0d9b9edd3b3f127a6bfa660baabc2e2774a794ab649a12" Jul 18 16:37:02.416534 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.416490 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:02.416630 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:02.416618 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:03.276639 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.276561 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:03.276800 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.276692 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:37:03.277101 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.277079 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:03.277210 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.277181 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:03.277384 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.277368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:03.277472 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.277454 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:03.405668 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.405635 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="9a1be9bb926aa5a071fa48f98be0fc45540e7856753f80e31b78e8bafb1189b7" exitCode=0 Jul 18 16:37:03.406092 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.405719 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"9a1be9bb926aa5a071fa48f98be0fc45540e7856753f80e31b78e8bafb1189b7"} Jul 18 16:37:03.409424 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.409399 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:37:03.409812 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.409788 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" event={"ID":"72d5a44e-acb8-4ab5-99d3-ddd9c36704df","Type":"ContainerStarted","Data":"a8f50fdb9cbdae3709c94c3d06a166e5b45f9a12e8f51ae6faf0bb1b9b71099b"} Jul 18 16:37:03.450588 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.450540 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" podStartSLOduration=9.620912817 podStartE2EDuration="28.450525038s" totalImagesPullingTime="18.829612221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:03.449057447 +0000 UTC m=+28.587820131" watchObservedRunningTime="2026-07-18 16:37:03.450525038 +0000 UTC m=+28.589287692" Jul 18 16:37:03.959334 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.959302 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sc698"] Jul 18 16:37:03.959573 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.959459 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:03.959649 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.959596 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:37:03.962229 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.962201 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gzzf2"] Jul 18 16:37:03.962367 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.962348 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:03.962577 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.962546 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:03.962946 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.962924 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dqnmp"] Jul 18 16:37:03.963039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:03.963017 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:03.963118 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:03.963095 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:04.413969 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:04.413793 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="eec294ec3a848121376a6f9077348df9ad30d2fe398029ae9845941f39845609" exitCode=0 Jul 18 16:37:04.414295 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:04.413879 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"eec294ec3a848121376a6f9077348df9ad30d2fe398029ae9845941f39845609"} Jul 18 16:37:05.275075 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:05.275049 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:05.275217 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:05.275163 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:05.275217 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:05.275189 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:05.275316 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:05.275234 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:05.418448 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:05.418412 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="d16f0aa571df8f5a0f3d6508570ec6f240c613baf8fce3c85973810bb8961fc2" exitCode=0 Jul 18 16:37:05.418866 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:05.418473 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"d16f0aa571df8f5a0f3d6508570ec6f240c613baf8fce3c85973810bb8961fc2"} Jul 18 16:37:06.274356 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:06.274324 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:06.274538 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:06.274449 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:37:07.274280 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:07.274249 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:07.274932 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:07.274288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:07.274932 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:07.274403 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:07.274932 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:07.274492 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:08.274186 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:08.274156 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:08.274335 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:08.274269 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc698" podUID="c8a69b23-4563-44fd-825b-d3a148575aec" Jul 18 16:37:09.040451 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.040413 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:09.040651 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.040475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:09.040651 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.040587 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 16:37:09.040765 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.040653 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret podName:c8a69b23-4563-44fd-825b-d3a148575aec nodeName:}" failed. No retries permitted until 2026-07-18 16:37:41.040629322 +0000 UTC m=+66.179392000 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret") pod "global-pull-secret-syncer-sc698" (UID: "c8a69b23-4563-44fd-825b-d3a148575aec") : object "kube-system"/"original-pull-secret" not registered Jul 18 16:37:09.040765 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.040592 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:37:09.040849 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.040764 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:37:41.040743664 +0000 UTC m=+66.179506321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 16:37:09.141835 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.141802 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:09.141984 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.141921 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 16:37:09.141984 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.141942 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 16:37:09.141984 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.141955 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xjhtf for pod openshift-network-diagnostics/network-check-target-dqnmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:37:09.142113 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.142006 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf podName:07ed0124-f75b-4c76-aae8-f47299ad8d11 nodeName:}" failed. No retries permitted until 2026-07-18 16:37:41.141992819 +0000 UTC m=+66.280755473 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xjhtf" (UniqueName: "kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf") pod "network-check-target-dqnmp" (UID: "07ed0124-f75b-4c76-aae8-f47299ad8d11") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 16:37:09.274079 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.274050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:09.274246 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.274050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:09.274246 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.274180 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dqnmp" podUID="07ed0124-f75b-4c76-aae8-f47299ad8d11" Jul 18 16:37:09.274358 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:09.274272 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:37:09.745476 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.745451 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-1.ec2.internal" event="NodeReady" Jul 18 16:37:09.745631 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.745591 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 16:37:09.786033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.785903 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qmdfl"] Jul 18 16:37:09.788915 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.788895 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fzrhq"] Jul 18 16:37:09.789034 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.789018 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:09.791385 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.791356 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 16:37:09.791543 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.791494 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 16:37:09.791650 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.791593 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:09.791650 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.791621 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jrkz\"" Jul 18 16:37:09.793726 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.793708 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 16:37:09.793816 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.793728 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 16:37:09.793816 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.793713 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 16:37:09.793816 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.793783 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5r7jb\"" Jul 18 16:37:09.798772 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.798753 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qmdfl"] Jul 18 16:37:09.802254 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.802063 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fzrhq"] Jul 18 16:37:09.946895 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.946865 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znzrq\" (UniqueName: \"kubernetes.io/projected/64bc4b28-2f6c-4b23-93e2-baab4394473c-kube-api-access-znzrq\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:09.947010 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.946911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64bc4b28-2f6c-4b23-93e2-baab4394473c-config-volume\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:09.947047 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.947006 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64bc4b28-2f6c-4b23-93e2-baab4394473c-tmp-dir\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:09.947094 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.947062 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4llk\" (UniqueName: \"kubernetes.io/projected/044fcfca-f17d-4ee2-887c-add500469cdf-kube-api-access-c4llk\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:09.947094 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.947087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:09.947166 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:09.947111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/044fcfca-f17d-4ee2-887c-add500469cdf-cert\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.048146 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4llk\" (UniqueName: \"kubernetes.io/projected/044fcfca-f17d-4ee2-887c-add500469cdf-kube-api-access-c4llk\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.048274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.048274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048180 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/044fcfca-f17d-4ee2-887c-add500469cdf-cert\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.048274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048204 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znzrq\" (UniqueName: \"kubernetes.io/projected/64bc4b28-2f6c-4b23-93e2-baab4394473c-kube-api-access-znzrq\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.048274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048228 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64bc4b28-2f6c-4b23-93e2-baab4394473c-config-volume\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.048424 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:10.048280 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:10.048424 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:10.048341 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:10.54832553 +0000 UTC m=+35.687088172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:10.048424 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64bc4b28-2f6c-4b23-93e2-baab4394473c-tmp-dir\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.048695 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048674 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64bc4b28-2f6c-4b23-93e2-baab4394473c-tmp-dir\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.048836 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.048819 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64bc4b28-2f6c-4b23-93e2-baab4394473c-config-volume\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.052162 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.052141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/044fcfca-f17d-4ee2-887c-add500469cdf-cert\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.058464 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.058438 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znzrq\" (UniqueName: \"kubernetes.io/projected/64bc4b28-2f6c-4b23-93e2-baab4394473c-kube-api-access-znzrq\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.058689 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.058674 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4llk\" (UniqueName: \"kubernetes.io/projected/044fcfca-f17d-4ee2-887c-add500469cdf-kube-api-access-c4llk\") pod \"ingress-canary-fzrhq\" (UID: \"044fcfca-f17d-4ee2-887c-add500469cdf\") " pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.106711 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.106649 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fzrhq" Jul 18 16:37:10.236152 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.236000 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fzrhq"] Jul 18 16:37:10.239139 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:37:10.239112 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044fcfca_f17d_4ee2_887c_add500469cdf.slice/crio-11f2c45f6887e6baf80f3a6adde98c3cbb6c1e8304423c2c6e586cd51a09189e WatchSource:0}: Error finding container 11f2c45f6887e6baf80f3a6adde98c3cbb6c1e8304423c2c6e586cd51a09189e: Status 404 returned error can't find the container with id 11f2c45f6887e6baf80f3a6adde98c3cbb6c1e8304423c2c6e586cd51a09189e Jul 18 16:37:10.274740 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.274720 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:10.277611 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.277596 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 16:37:10.434224 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.434193 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="735594bbb6a1fc24797c685c13d227b64b5ec8db86cc40335c4fc82bc58aaf6e" exitCode=0 Jul 18 16:37:10.434380 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.434268 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"735594bbb6a1fc24797c685c13d227b64b5ec8db86cc40335c4fc82bc58aaf6e"} Jul 18 16:37:10.435405 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.435385 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fzrhq" event={"ID":"044fcfca-f17d-4ee2-887c-add500469cdf","Type":"ContainerStarted","Data":"11f2c45f6887e6baf80f3a6adde98c3cbb6c1e8304423c2c6e586cd51a09189e"} Jul 18 16:37:10.553064 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:10.553033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:10.553177 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:10.553157 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:10.553253 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:10.553234 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:11.553212334 +0000 UTC m=+36.691974999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:11.274907 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.274868 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:11.275384 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.275076 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:11.279191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.279027 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 16:37:11.279191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.279045 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 16:37:11.279191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.279086 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pm4pn\"" Jul 18 16:37:11.279191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.279114 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 16:37:11.279191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.279027 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9vx5\"" Jul 18 16:37:11.440564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.440534 2567 generic.go:356] "Generic (PLEG): container finished" podID="d9291fc5-ce9c-4fec-a60a-27ec7a2ba957" containerID="60bdc9c92e64962e8228e6056faa59f41d6f442acab551f47924517a44e60729" exitCode=0 Jul 18 16:37:11.440564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.440568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerDied","Data":"60bdc9c92e64962e8228e6056faa59f41d6f442acab551f47924517a44e60729"} Jul 18 16:37:11.561168 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:11.561137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:11.561343 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:11.561325 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:11.561418 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:11.561400 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:13.561378923 +0000 UTC m=+38.700141583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:12.446150 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:12.446002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hswxz" event={"ID":"d9291fc5-ce9c-4fec-a60a-27ec7a2ba957","Type":"ContainerStarted","Data":"ebf55cfce234079c75bc1874d3e24546254ae97cb108802489216dbef6515947"} Jul 18 16:37:12.478068 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:12.476636 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hswxz" podStartSLOduration=7.8831089819999995 podStartE2EDuration="37.476617998s" totalImagesPullingTime="29.593509016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:12.476313158 +0000 UTC m=+37.615075823" watchObservedRunningTime="2026-07-18 16:37:12.476617998 +0000 UTC m=+37.615380661" Jul 18 16:37:13.449283 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:13.449251 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fzrhq" event={"ID":"044fcfca-f17d-4ee2-887c-add500469cdf","Type":"ContainerStarted","Data":"b8ae89b989c69e9dc7062df8e31ed6df7746b4ec11c7d1e84aca7d370bf915ae"} Jul 18 16:37:13.467269 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:13.467226 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fzrhq" podStartSLOduration=2.264074868 podStartE2EDuration="4.467211591s" totalImagesPullingTime="2.203136723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:13.466953486 +0000 UTC m=+38.605716162" watchObservedRunningTime="2026-07-18 16:37:13.467211591 +0000 UTC m=+38.605974253" Jul 18 16:37:13.575091 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:13.575053 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:13.575239 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:13.575154 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:13.575239 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:13.575206 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:17.575190078 +0000 UTC m=+42.713952733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:17.601818 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:17.601780 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:17.602183 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:17.601901 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:17.602183 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:17.601952 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:25.60193924 +0000 UTC m=+50.740701881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:25.661345 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:25.661307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:25.661823 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:25.661419 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:25.661823 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:25.661469 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:37:41.661456632 +0000 UTC m=+66.800219277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:34.433991 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:34.433965 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smlc5" Jul 18 16:37:41.065718 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.065673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:37:41.065718 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.065724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:41.068708 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.068689 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 16:37:41.068774 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.068758 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 16:37:41.076739 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:41.076717 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 18 16:37:41.076826 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:41.076814 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:38:45.076792636 +0000 UTC m=+130.215555280 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : secret "metrics-daemon-secret" not found Jul 18 16:37:41.080402 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.080375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8a69b23-4563-44fd-825b-d3a148575aec-original-pull-secret\") pod \"global-pull-secret-syncer-sc698\" (UID: \"c8a69b23-4563-44fd-825b-d3a148575aec\") " pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:41.166583 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.166551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:41.169725 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.169710 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 16:37:41.179368 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.179353 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 16:37:41.183666 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.183653 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc698" Jul 18 16:37:41.190154 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.190128 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhtf\" (UniqueName: \"kubernetes.io/projected/07ed0124-f75b-4c76-aae8-f47299ad8d11-kube-api-access-xjhtf\") pod \"network-check-target-dqnmp\" (UID: \"07ed0124-f75b-4c76-aae8-f47299ad8d11\") " pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:41.293390 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.293362 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9vx5\"" Jul 18 16:37:41.301541 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.301500 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:41.331247 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.331223 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sc698"] Jul 18 16:37:41.336537 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:37:41.336309 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a69b23_4563_44fd_825b_d3a148575aec.slice/crio-b3907398f73cf1b394291701d4c85001ceb4b5ff3f3ba2135015e79e343a3036 WatchSource:0}: Error finding container b3907398f73cf1b394291701d4c85001ceb4b5ff3f3ba2135015e79e343a3036: Status 404 returned error can't find the container with id b3907398f73cf1b394291701d4c85001ceb4b5ff3f3ba2135015e79e343a3036 Jul 18 16:37:41.432305 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.432278 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dqnmp"] Jul 18 16:37:41.435292 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:37:41.435262 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ed0124_f75b_4c76_aae8_f47299ad8d11.slice/crio-550bec3868bb6b9d643e7199a9b1059995313fdcd8661eb960eeb21c02ef8b80 WatchSource:0}: Error finding container 550bec3868bb6b9d643e7199a9b1059995313fdcd8661eb960eeb21c02ef8b80: Status 404 returned error can't find the container with id 550bec3868bb6b9d643e7199a9b1059995313fdcd8661eb960eeb21c02ef8b80 Jul 18 16:37:41.513417 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.513383 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sc698" event={"ID":"c8a69b23-4563-44fd-825b-d3a148575aec","Type":"ContainerStarted","Data":"b3907398f73cf1b394291701d4c85001ceb4b5ff3f3ba2135015e79e343a3036"} Jul 18 16:37:41.514265 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.514241 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dqnmp" event={"ID":"07ed0124-f75b-4c76-aae8-f47299ad8d11","Type":"ContainerStarted","Data":"550bec3868bb6b9d643e7199a9b1059995313fdcd8661eb960eeb21c02ef8b80"} Jul 18 16:37:41.670835 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:41.670806 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:37:41.670967 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:41.670935 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:37:41.671007 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:37:41.670997 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:38:13.670983411 +0000 UTC m=+98.809746055 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:37:46.531616 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:46.531582 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sc698" event={"ID":"c8a69b23-4563-44fd-825b-d3a148575aec","Type":"ContainerStarted","Data":"410b48228b067617f022f25abfdab3daa024d94e8bd738bc99b8880a91281613"} Jul 18 16:37:46.533024 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:46.532997 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dqnmp" event={"ID":"07ed0124-f75b-4c76-aae8-f47299ad8d11","Type":"ContainerStarted","Data":"a83159bb8c6c5b3cc3d69c28ad45418790ff467a014392c92ec5204bc89b0ef0"} Jul 18 16:37:46.533146 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:46.533128 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:37:46.545690 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:46.545646 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sc698" podStartSLOduration=66.985867787 podStartE2EDuration="1m11.545629037s" totalImagesPullingTime="4.55976125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:46.545577642 +0000 UTC m=+71.684340305" watchObservedRunningTime="2026-07-18 16:37:46.545629037 +0000 UTC m=+71.684391701" Jul 18 16:37:46.561561 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:37:46.561497 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dqnmp" podStartSLOduration=67.102963019 podStartE2EDuration="1m11.561481286s" totalImagesPullingTime="4.458518267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:37:46.560868641 +0000 UTC m=+71.699631305" watchObservedRunningTime="2026-07-18 16:37:46.561481286 +0000 UTC m=+71.700243949" Jul 18 16:38:13.680475 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:13.680425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:38:13.680893 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:38:13.680607 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:38:13.680893 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:38:13.680686 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:39:17.680669936 +0000 UTC m=+162.819432578 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:38:17.538399 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:17.538367 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dqnmp" Jul 18 16:38:19.911059 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.911024 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv"] Jul 18 16:38:19.914599 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.914580 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:19.917271 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.917252 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 16:38:19.918305 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.918286 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 18 16:38:19.918390 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.918304 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 16:38:19.918390 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.918289 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 16:38:19.918390 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.918331 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cr7ts\"" Jul 18 16:38:19.922235 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.922214 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv"] Jul 18 16:38:19.935952 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.935926 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd"] Jul 18 16:38:19.938396 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.938379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:19.941383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.941354 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 18 16:38:19.941383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.941378 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 18 16:38:19.941525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.941470 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 18 16:38:19.941679 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.941668 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 18 16:38:19.950847 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:19.950830 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd"] Jul 18 16:38:20.021151 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.021251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.021251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr4np\" (UniqueName: \"kubernetes.io/projected/d76e5486-bee4-463f-a71f-0318e6bf67f0-kube-api-access-gr4np\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.021251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021208 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.021251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021223 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c466d066-b861-4e1e-af69-2cd10e0018b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.021251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021244 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwn54\" (UniqueName: \"kubernetes.io/projected/c466d066-b861-4e1e-af69-2cd10e0018b6-kube-api-access-nwn54\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.021423 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021261 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.021423 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.021280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76e5486-bee4-463f-a71f-0318e6bf67f0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122548 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76e5486-bee4-463f-a71f-0318e6bf67f0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122568 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr4np\" (UniqueName: \"kubernetes.io/projected/d76e5486-bee4-463f-a71f-0318e6bf67f0-kube-api-access-gr4np\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.122683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122653 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c466d066-b861-4e1e-af69-2cd10e0018b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.122899 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122693 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwn54\" (UniqueName: \"kubernetes.io/projected/c466d066-b861-4e1e-af69-2cd10e0018b6-kube-api-access-nwn54\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.122899 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.122726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.123410 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.123384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76e5486-bee4-463f-a71f-0318e6bf67f0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.125292 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.125264 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.125388 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.125342 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.125430 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.125412 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-ca\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.125478 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.125462 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c466d066-b861-4e1e-af69-2cd10e0018b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.125536 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.125488 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d76e5486-bee4-463f-a71f-0318e6bf67f0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.131590 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.131570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr4np\" (UniqueName: \"kubernetes.io/projected/d76e5486-bee4-463f-a71f-0318e6bf67f0-kube-api-access-gr4np\") pod \"cluster-proxy-proxy-agent-7b58544ff6-qq4wd\" (UID: \"d76e5486-bee4-463f-a71f-0318e6bf67f0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.132182 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.132159 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwn54\" (UniqueName: \"kubernetes.io/projected/c466d066-b861-4e1e-af69-2cd10e0018b6-kube-api-access-nwn54\") pod \"managed-serviceaccount-addon-agent-6b799cccc4-bnrlv\" (UID: \"c466d066-b861-4e1e-af69-2cd10e0018b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.235953 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.235890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" Jul 18 16:38:20.246583 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.246557 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:38:20.362805 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.362775 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv"] Jul 18 16:38:20.365953 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:38:20.365929 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc466d066_b861_4e1e_af69_2cd10e0018b6.slice/crio-707d7f3634c8f2fa7bffd2823ee67e5c56e3d55797944c6dd84e9b212362d537 WatchSource:0}: Error finding container 707d7f3634c8f2fa7bffd2823ee67e5c56e3d55797944c6dd84e9b212362d537: Status 404 returned error can't find the container with id 707d7f3634c8f2fa7bffd2823ee67e5c56e3d55797944c6dd84e9b212362d537 Jul 18 16:38:20.378418 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.378391 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd"] Jul 18 16:38:20.381716 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:38:20.381685 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd76e5486_bee4_463f_a71f_0318e6bf67f0.slice/crio-cfa12f3ef455ba90f78920f36fe695fee97eaecafe20c4706f7573f86fda3be6 WatchSource:0}: Error finding container cfa12f3ef455ba90f78920f36fe695fee97eaecafe20c4706f7573f86fda3be6: Status 404 returned error can't find the container with id cfa12f3ef455ba90f78920f36fe695fee97eaecafe20c4706f7573f86fda3be6 Jul 18 16:38:20.612144 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.612064 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerStarted","Data":"cfa12f3ef455ba90f78920f36fe695fee97eaecafe20c4706f7573f86fda3be6"} Jul 18 16:38:20.613019 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:20.612994 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" event={"ID":"c466d066-b861-4e1e-af69-2cd10e0018b6","Type":"ContainerStarted","Data":"707d7f3634c8f2fa7bffd2823ee67e5c56e3d55797944c6dd84e9b212362d537"} Jul 18 16:38:24.628204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:24.627879 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" event={"ID":"c466d066-b861-4e1e-af69-2cd10e0018b6","Type":"ContainerStarted","Data":"95cbd37f6d332518bb29480817af4beef1aa2052bfb88653e98e486e58c77814"} Jul 18 16:38:24.630079 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:24.630053 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerStarted","Data":"b25dab651e4696693c1e0b3fb9a793d82ffcc343d38fef8474ade31f4437bdc3"} Jul 18 16:38:24.641753 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:24.641710 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" podStartSLOduration=1.5094458899999998 podStartE2EDuration="5.641694559s" totalImagesPullingTime="4.132248669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:38:24.64075549 +0000 UTC m=+109.779518154" watchObservedRunningTime="2026-07-18 16:38:24.641694559 +0000 UTC m=+109.780457224" Jul 18 16:38:25.635044 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:25.635008 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerStarted","Data":"28febc2f4cfee38502a4fc60aab54926b1987133a154a691809ddc30f56e191d"} Jul 18 16:38:25.635044 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:25.635052 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerStarted","Data":"a0a4bb9939b31f828ae8a89c8a0be0764a756a9523963dcad4dc7191967587d4"} Jul 18 16:38:25.653383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:25.653327 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" podStartSLOduration=2.525155796 podStartE2EDuration="6.653313917s" totalImagesPullingTime="4.128158121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:38:25.652004647 +0000 UTC m=+110.790767330" watchObservedRunningTime="2026-07-18 16:38:25.653313917 +0000 UTC m=+110.792076580" Jul 18 16:38:45.099052 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:38:45.099013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:38:45.099453 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:38:45.099126 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 18 16:38:45.099453 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:38:45.099184 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:47.09916884 +0000 UTC m=+252.237931481 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : secret "metrics-daemon-secret" not found Jul 18 16:39:12.800276 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:39:12.800220 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qmdfl" podUID="64bc4b28-2f6c-4b23-93e2-baab4394473c" Jul 18 16:39:13.768956 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:39:13.768879 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:39:14.287278 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:39:14.287237 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:39:17.726429 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:39:17.726392 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:39:17.726834 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:39:17.726500 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 16:39:17.726834 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:39:17.726580 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls podName:64bc4b28-2f6c-4b23-93e2-baab4394473c nodeName:}" failed. No retries permitted until 2026-07-18 16:41:19.726565298 +0000 UTC m=+284.865327938 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls") pod "dns-default-qmdfl" (UID: "64bc4b28-2f6c-4b23-93e2-baab4394473c") : secret "dns-default-metrics-tls" not found Jul 18 16:39:26.274324 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:39:26.274283 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:40:24.965528 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:24.965422 2567 generic.go:356] "Generic (PLEG): container finished" podID="c466d066-b861-4e1e-af69-2cd10e0018b6" containerID="95cbd37f6d332518bb29480817af4beef1aa2052bfb88653e98e486e58c77814" exitCode=255 Jul 18 16:40:24.965528 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:24.965496 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" event={"ID":"c466d066-b861-4e1e-af69-2cd10e0018b6","Type":"ContainerDied","Data":"95cbd37f6d332518bb29480817af4beef1aa2052bfb88653e98e486e58c77814"} Jul 18 16:40:24.965968 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:24.965828 2567 scope.go:117] "RemoveContainer" containerID="95cbd37f6d332518bb29480817af4beef1aa2052bfb88653e98e486e58c77814" Jul 18 16:40:25.970367 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:25.970334 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b799cccc4-bnrlv" event={"ID":"c466d066-b861-4e1e-af69-2cd10e0018b6","Type":"ContainerStarted","Data":"d526c39e18db2a24c16fc2317ec5b834c81013fbe820380adcd19c2a6c8669c4"} Jul 18 16:40:35.630937 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.630901 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x4p87"] Jul 18 16:40:35.633866 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.633848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.636247 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.636225 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 16:40:35.636348 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.636277 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6m7m\"" Jul 18 16:40:35.637295 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.637275 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 16:40:35.637358 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.637279 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 16:40:35.637358 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.637346 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 16:40:35.650903 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.650883 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x4p87"] Jul 18 16:40:35.737020 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.736994 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa6655f-6638-4835-b15e-c1965a3eefd4-crio-socket\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.737207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.737039 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.737207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.737107 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa6655f-6638-4835-b15e-c1965a3eefd4-data-volume\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.737207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.737148 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsjmx\" (UniqueName: \"kubernetes.io/projected/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-api-access-nsjmx\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.737207 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.737183 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838098 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838098 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838102 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa6655f-6638-4835-b15e-c1965a3eefd4-crio-socket\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838136 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838155 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa6655f-6638-4835-b15e-c1965a3eefd4-data-volume\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:35.838158 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsjmx\" (UniqueName: \"kubernetes.io/projected/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-api-access-nsjmx\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:35.838217 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:36.338204055 +0000 UTC m=+241.476966696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : configmap "kube-rbac-proxy" not found Jul 18 16:40:35.838274 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838250 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa6655f-6638-4835-b15e-c1965a3eefd4-crio-socket\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.838577 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:35.838389 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:35.838577 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:35.838438 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:36.338426509 +0000 UTC m=+241.477189153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:35.838577 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.838436 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa6655f-6638-4835-b15e-c1965a3eefd4-data-volume\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:35.847880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:35.847858 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsjmx\" (UniqueName: \"kubernetes.io/projected/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-api-access-nsjmx\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:36.342345 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:36.342292 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:36.342569 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:36.342417 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 16:40:36.342569 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:36.342450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:36.342569 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:36.342480 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:37.342465342 +0000 UTC m=+242.481227986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : configmap "kube-rbac-proxy" not found Jul 18 16:40:36.342569 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:36.342567 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:36.342756 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:36.342634 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:37.342619089 +0000 UTC m=+242.481381733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:37.351218 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:37.351164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:37.351675 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:37.351263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:37.351675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:37.351325 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:37.351675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:37.351358 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 16:40:37.351675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:37.351389 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:39.351373737 +0000 UTC m=+244.490136377 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:37.351675 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:37.351422 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:39.351405884 +0000 UTC m=+244.490168530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : configmap "kube-rbac-proxy" not found Jul 18 16:40:37.804292 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:37.804262 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tz5zg_940fc19a-b412-455e-ba83-285f02e11730/dns-node-resolver/0.log" Jul 18 16:40:38.404044 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:38.404021 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4s8cp_5379291e-07ab-4173-83da-9ee82abd6e42/node-ca/0.log" Jul 18 16:40:39.195479 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.195440 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-x9d78"] Jul 18 16:40:39.198439 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.198416 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.200690 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.200669 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 18 16:40:39.200946 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.200932 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 18 16:40:39.201699 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.201676 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t95jf\"" Jul 18 16:40:39.201784 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.201702 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 18 16:40:39.201784 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.201725 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 18 16:40:39.208540 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.208500 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-x9d78"] Jul 18 16:40:39.265579 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.265541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-cabundle\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.265744 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.265597 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-key\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.265744 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.265627 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nptwk\" (UniqueName: \"kubernetes.io/projected/90edf96b-9fff-43cd-b241-6deea422a9ee-kube-api-access-nptwk\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.366199 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366145 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:39.366199 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-key\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nptwk\" (UniqueName: \"kubernetes.io/projected/90edf96b-9fff-43cd-b241-6deea422a9ee-kube-api-access-nptwk\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:39.366260 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366278 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:39.366320 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:43.366306032 +0000 UTC m=+248.505068673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : configmap "kube-rbac-proxy" not found Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366339 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-cabundle\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:39.366375 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:39.366456 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:39.366425 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:43.366410257 +0000 UTC m=+248.505172898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:39.367016 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.366998 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-cabundle\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.368499 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.368479 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/90edf96b-9fff-43cd-b241-6deea422a9ee-signing-key\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.374585 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.374565 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nptwk\" (UniqueName: \"kubernetes.io/projected/90edf96b-9fff-43cd-b241-6deea422a9ee-kube-api-access-nptwk\") pod \"service-ca-58598695f7-x9d78\" (UID: \"90edf96b-9fff-43cd-b241-6deea422a9ee\") " pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.403887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.403869 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fzrhq_044fcfca-f17d-4ee2-887c-add500469cdf/serve-healthcheck-canary/0.log" Jul 18 16:40:39.507610 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.507537 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-x9d78" Jul 18 16:40:39.627499 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:39.627472 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-x9d78"] Jul 18 16:40:39.630711 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:40:39.630679 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90edf96b_9fff_43cd_b241_6deea422a9ee.slice/crio-2ba3de327b0fd8c3ec01dfc12ba3bc9e9f3584257ee1111be52495db6da7f32e WatchSource:0}: Error finding container 2ba3de327b0fd8c3ec01dfc12ba3bc9e9f3584257ee1111be52495db6da7f32e: Status 404 returned error can't find the container with id 2ba3de327b0fd8c3ec01dfc12ba3bc9e9f3584257ee1111be52495db6da7f32e Jul 18 16:40:40.008051 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:40.008016 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-x9d78" event={"ID":"90edf96b-9fff-43cd-b241-6deea422a9ee","Type":"ContainerStarted","Data":"2ba3de327b0fd8c3ec01dfc12ba3bc9e9f3584257ee1111be52495db6da7f32e"} Jul 18 16:40:42.015733 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:42.015700 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-x9d78" event={"ID":"90edf96b-9fff-43cd-b241-6deea422a9ee","Type":"ContainerStarted","Data":"4c64079043eac70cdfdc51f2e35afdcc60d9380001e7852e548913e1947f3fed"} Jul 18 16:40:42.032524 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:42.032468 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-x9d78" podStartSLOduration=1.446919471 podStartE2EDuration="3.032452921s" totalImagesPullingTime="1.58553345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:40:42.031853661 +0000 UTC m=+247.170616325" watchObservedRunningTime="2026-07-18 16:40:42.032452921 +0000 UTC m=+247.171215585" Jul 18 16:40:43.402895 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:43.402855 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 16:40:43.403317 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:43.402952 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:51.40293248 +0000 UTC m=+256.541695129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : configmap "kube-rbac-proxy" not found Jul 18 16:40:43.403317 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:43.402860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:43.403317 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:43.403048 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:43.403317 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:43.403162 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:43.403317 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:43.403240 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:40:51.403223165 +0000 UTC m=+256.541985812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:47.132053 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:47.132008 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:40:47.132562 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:47.132154 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 18 16:40:47.132562 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:47.132209 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs podName:63a253e8-58a9-4e0d-a5a6-3fea89b67a56 nodeName:}" failed. No retries permitted until 2026-07-18 16:42:49.132195099 +0000 UTC m=+374.270957743 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs") pod "network-metrics-daemon-gzzf2" (UID: "63a253e8-58a9-4e0d-a5a6-3fea89b67a56") : secret "metrics-daemon-secret" not found Jul 18 16:40:51.465357 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:51.465319 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:51.465785 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:51.465405 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:40:51.465785 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:51.465471 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 16:40:51.465785 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:40:51.465553 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls podName:4fa6655f-6638-4835-b15e-c1965a3eefd4 nodeName:}" failed. No retries permitted until 2026-07-18 16:41:07.465535772 +0000 UTC m=+272.604298422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x4p87" (UID: "4fa6655f-6638-4835-b15e-c1965a3eefd4") : secret "insights-runtime-extractor-tls" not found Jul 18 16:40:51.465916 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:40:51.465868 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa6655f-6638-4835-b15e-c1965a3eefd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:41:00.247860 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:00.247770 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" podUID="d76e5486-bee4-463f-a71f-0318e6bf67f0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 18 16:41:07.480582 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:07.480532 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:41:07.482906 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:07.482879 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa6655f-6638-4835-b15e-c1965a3eefd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4p87\" (UID: \"4fa6655f-6638-4835-b15e-c1965a3eefd4\") " pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:41:07.742136 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:07.742035 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x4p87" Jul 18 16:41:07.862868 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:07.862840 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x4p87"] Jul 18 16:41:07.865401 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:41:07.865378 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa6655f_6638_4835_b15e_c1965a3eefd4.slice/crio-ab4fd4ddf393a44354f40d12e587a5713fbc4af3ef66e8447e99cdb950f5ecff WatchSource:0}: Error finding container ab4fd4ddf393a44354f40d12e587a5713fbc4af3ef66e8447e99cdb950f5ecff: Status 404 returned error can't find the container with id ab4fd4ddf393a44354f40d12e587a5713fbc4af3ef66e8447e99cdb950f5ecff Jul 18 16:41:08.089298 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:08.089213 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4p87" event={"ID":"4fa6655f-6638-4835-b15e-c1965a3eefd4","Type":"ContainerStarted","Data":"ab4fd4ddf393a44354f40d12e587a5713fbc4af3ef66e8447e99cdb950f5ecff"} Jul 18 16:41:10.248117 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:10.248057 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" podUID="d76e5486-bee4-463f-a71f-0318e6bf67f0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 18 16:41:13.107774 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:13.107687 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4p87" event={"ID":"4fa6655f-6638-4835-b15e-c1965a3eefd4","Type":"ContainerStarted","Data":"42a8956d66ee54fa35ecbdb6cdf97f2e5862c8385ecfb03b8d491bba3934e794"} Jul 18 16:41:16.120531 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:16.120476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4p87" event={"ID":"4fa6655f-6638-4835-b15e-c1965a3eefd4","Type":"ContainerStarted","Data":"da25fd7cf961305b62d55c5e71c712e104f158b0c97f32d05436f4470b811942"} Jul 18 16:41:16.769439 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:41:16.769397 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qmdfl" podUID="64bc4b28-2f6c-4b23-93e2-baab4394473c" Jul 18 16:41:17.123645 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:17.123611 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:18.349215 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.348890 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xlbmx"] Jul 18 16:41:18.361432 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.361404 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.364225 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.363863 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 16:41:18.364225 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.364094 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8ss7d\"" Jul 18 16:41:18.365306 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.364829 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 16:41:18.366478 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.366448 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 16:41:18.366647 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.366603 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 16:41:18.366711 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.366669 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 16:41:18.366788 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.366763 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 16:41:18.463739 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.463701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.463924 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.463767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-wtmp\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.463924 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.463846 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-sys\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.463924 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.463914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvzs\" (UniqueName: \"kubernetes.io/projected/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-kube-api-access-ghvzs\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.464149 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.463942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.464149 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.464009 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-metrics-client-ca\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.464149 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.464067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-textfile\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.464149 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.464097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-root\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.464149 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.464126 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.564851 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.564803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-wtmp\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.564859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-sys\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.564932 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-sys\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.564998 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-wtmp\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565003 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvzs\" (UniqueName: \"kubernetes.io/projected/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-kube-api-access-ghvzs\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565054 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565109 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-metrics-client-ca\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-textfile\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565161 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-root\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565196 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565237 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565412 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:41:18.565385 2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 16:41:18.565833 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:41:18.565442 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls podName:03f0e8a5-651a-4464-9dd4-f725eda6bd5c nodeName:}" failed. No retries permitted until 2026-07-18 16:41:19.065423088 +0000 UTC m=+284.204185743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls") pod "node-exporter-xlbmx" (UID: "03f0e8a5-651a-4464-9dd4-f725eda6bd5c") : secret "node-exporter-tls" not found Jul 18 16:41:18.565833 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565655 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-textfile\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.565833 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.565701 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-root\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.566223 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.566182 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.566545 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.566493 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-metrics-client-ca\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.568586 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.568550 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:18.573334 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:18.573316 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvzs\" (UniqueName: \"kubernetes.io/projected/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-kube-api-access-ghvzs\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:19.069576 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.069543 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:19.072141 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.072114 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03f0e8a5-651a-4464-9dd4-f725eda6bd5c-node-exporter-tls\") pod \"node-exporter-xlbmx\" (UID: \"03f0e8a5-651a-4464-9dd4-f725eda6bd5c\") " pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:19.274308 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.274270 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlbmx" Jul 18 16:41:19.386790 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.386756 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:41:19.394173 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.394148 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.396734 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.396856 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.396895 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p9fdm\"" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.396906 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.396942 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 18 16:41:19.397033 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.397035 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 18 16:41:19.397383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.397062 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 18 16:41:19.397383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.397221 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 18 16:41:19.397383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.397239 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 18 16:41:19.397474 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.397455 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 18 16:41:19.405333 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.405308 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:41:19.473941 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.473911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474099 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.473950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474099 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.473970 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474099 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474216 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474134 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55rz8\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474216 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474216 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474216 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474237 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474275 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474291 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.474352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.474347 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.541197 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:41:19.541160 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f0e8a5_651a_4464_9dd4_f725eda6bd5c.slice/crio-7768c360326a54d8d04cd7f5b4ee76c92becf03cf165bb1640feeb4e18c3674d WatchSource:0}: Error finding container 7768c360326a54d8d04cd7f5b4ee76c92becf03cf165bb1640feeb4e18c3674d: Status 404 returned error can't find the container with id 7768c360326a54d8d04cd7f5b4ee76c92becf03cf165bb1640feeb4e18c3674d Jul 18 16:41:19.574666 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.574775 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55rz8\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.574775 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574699 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.574905 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.574969 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575019 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.574970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575019 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575004 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575117 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575054 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575381 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575355 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575381 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575371 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575534 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575534 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575701 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575701 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575557 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575701 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.575664 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.575701 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:41:19.575678 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle podName:b867f2d8-deb4-41bd-89af-62d82eac68ca nodeName:}" failed. No retries permitted until 2026-07-18 16:41:20.075660718 +0000 UTC m=+285.214423374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca") : configmap references non-existent config key: ca-bundle.crt Jul 18 16:41:19.577877 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.577829 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578025 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578000 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578277 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578397 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578602 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578684 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578665 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.578940 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.578918 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.579299 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.579282 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.579527 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.579491 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.582679 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.582658 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55rz8\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:19.777876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.777836 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:19.780096 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.780073 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64bc4b28-2f6c-4b23-93e2-baab4394473c-metrics-tls\") pod \"dns-default-qmdfl\" (UID: \"64bc4b28-2f6c-4b23-93e2-baab4394473c\") " pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:19.826278 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.826248 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jrkz\"" Jul 18 16:41:19.834112 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.834085 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:19.957472 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:19.957439 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qmdfl"] Jul 18 16:41:19.960788 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:41:19.960768 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bc4b28_2f6c_4b23_93e2_baab4394473c.slice/crio-85075a3b6e440c8fc6ef0a6cca76915ef83ff2c3bef90759f8c2b9e13ff478bd WatchSource:0}: Error finding container 85075a3b6e440c8fc6ef0a6cca76915ef83ff2c3bef90759f8c2b9e13ff478bd: Status 404 returned error can't find the container with id 85075a3b6e440c8fc6ef0a6cca76915ef83ff2c3bef90759f8c2b9e13ff478bd Jul 18 16:41:20.081067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.081035 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:20.081780 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.081759 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:20.134869 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.134831 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4p87" event={"ID":"4fa6655f-6638-4835-b15e-c1965a3eefd4","Type":"ContainerStarted","Data":"1a6c6073d0558b52cb329d68c6fe4a4a492aba02b5130ee57492ec141cb34b8e"} Jul 18 16:41:20.135940 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.135912 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qmdfl" event={"ID":"64bc4b28-2f6c-4b23-93e2-baab4394473c","Type":"ContainerStarted","Data":"85075a3b6e440c8fc6ef0a6cca76915ef83ff2c3bef90759f8c2b9e13ff478bd"} Jul 18 16:41:20.137027 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.137007 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlbmx" event={"ID":"03f0e8a5-651a-4464-9dd4-f725eda6bd5c","Type":"ContainerStarted","Data":"7768c360326a54d8d04cd7f5b4ee76c92becf03cf165bb1640feeb4e18c3674d"} Jul 18 16:41:20.153065 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.153012 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x4p87" podStartSLOduration=33.541396926 podStartE2EDuration="45.152998073s" totalImagesPullingTime="11.611601147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 16:40:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:41:20.152026807 +0000 UTC m=+285.290789512" watchObservedRunningTime="2026-07-18 16:41:20.152998073 +0000 UTC m=+285.291760737" Jul 18 16:41:20.247940 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.247742 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" podUID="d76e5486-bee4-463f-a71f-0318e6bf67f0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 18 16:41:20.248152 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.248126 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" Jul 18 16:41:20.248769 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.248738 2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a0a4bb9939b31f828ae8a89c8a0be0764a756a9523963dcad4dc7191967587d4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 18 16:41:20.248877 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.248793 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" podUID="d76e5486-bee4-463f-a71f-0318e6bf67f0" containerName="service-proxy" containerID="cri-o://a0a4bb9939b31f828ae8a89c8a0be0764a756a9523963dcad4dc7191967587d4" gracePeriod=30 Jul 18 16:41:20.305697 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.305661 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:41:20.325610 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.325585 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7954b47d79-tl6wz"] Jul 18 16:41:20.329355 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.329332 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.333662 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.333633 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 18 16:41:20.333767 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.333670 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 18 16:41:20.333767 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.333684 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 18 16:41:20.334040 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.334020 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 18 16:41:20.334188 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.334172 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 18 16:41:20.334263 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.334174 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d4ap5eqb41jap\"" Jul 18 16:41:20.334263 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.334199 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tc4nd\"" Jul 18 16:41:20.346362 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.346332 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7954b47d79-tl6wz"] Jul 18 16:41:20.383958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383810 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.383958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383865 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.383958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383894 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gp8v\" (UniqueName: \"kubernetes.io/projected/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-kube-api-access-6gp8v\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.383958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383936 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.383958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383964 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-grpc-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.384284 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.383996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.384284 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.384058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-metrics-client-ca\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.384284 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.384095 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.472084 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.471913 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:41:20.485660 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-metrics-client-ca\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485660 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485642 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485778 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gp8v\" (UniqueName: \"kubernetes.io/projected/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-kube-api-access-6gp8v\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485819 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.485876 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-grpc-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.486129 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.485878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.486359 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.486335 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-metrics-client-ca\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.489287 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.488771 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.489287 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.489189 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.489287 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.489247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.489832 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.489807 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-grpc-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.489902 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.489883 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.490755 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.490730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-secret-thanos-querier-tls\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.496659 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.496633 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gp8v\" (UniqueName: \"kubernetes.io/projected/a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417-kube-api-access-6gp8v\") pod \"thanos-querier-7954b47d79-tl6wz\" (UID: \"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417\") " pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.640714 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.640674 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:20.670394 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:41:20.670358 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb867f2d8_deb4_41bd_89af_62d82eac68ca.slice/crio-e4006c4e1a54233e7238c285613bb876e0b7055e681dea9000d033182044544f WatchSource:0}: Error finding container e4006c4e1a54233e7238c285613bb876e0b7055e681dea9000d033182044544f: Status 404 returned error can't find the container with id e4006c4e1a54233e7238c285613bb876e0b7055e681dea9000d033182044544f Jul 18 16:41:20.823068 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:20.823039 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7954b47d79-tl6wz"] Jul 18 16:41:20.825989 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:41:20.825965 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ba530b_cd9c_4a7c_b080_2c4d8a6a2417.slice/crio-fd986ea8c7767bf9190642c2e0687fb1c9737a59b4ba910d314325acd9502f02 WatchSource:0}: Error finding container fd986ea8c7767bf9190642c2e0687fb1c9737a59b4ba910d314325acd9502f02: Status 404 returned error can't find the container with id fd986ea8c7767bf9190642c2e0687fb1c9737a59b4ba910d314325acd9502f02 Jul 18 16:41:21.145631 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.145586 2567 generic.go:356] "Generic (PLEG): container finished" podID="d76e5486-bee4-463f-a71f-0318e6bf67f0" containerID="a0a4bb9939b31f828ae8a89c8a0be0764a756a9523963dcad4dc7191967587d4" exitCode=2 Jul 18 16:41:21.145791 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.145660 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerDied","Data":"a0a4bb9939b31f828ae8a89c8a0be0764a756a9523963dcad4dc7191967587d4"} Jul 18 16:41:21.145791 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.145715 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b58544ff6-qq4wd" event={"ID":"d76e5486-bee4-463f-a71f-0318e6bf67f0","Type":"ContainerStarted","Data":"9fbc032de94439449e7f05498d1a8ffeea6cc94c03dc02f18866e4be36714b26"} Jul 18 16:41:21.147169 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.147140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"fd986ea8c7767bf9190642c2e0687fb1c9737a59b4ba910d314325acd9502f02"} Jul 18 16:41:21.149041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.148905 2567 generic.go:356] "Generic (PLEG): container finished" podID="03f0e8a5-651a-4464-9dd4-f725eda6bd5c" containerID="d1ffecad3414d71e27a83b4f539e5d90d8f6658cf772aa52d77fccdedb709f8c" exitCode=0 Jul 18 16:41:21.149041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.148987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlbmx" event={"ID":"03f0e8a5-651a-4464-9dd4-f725eda6bd5c","Type":"ContainerDied","Data":"d1ffecad3414d71e27a83b4f539e5d90d8f6658cf772aa52d77fccdedb709f8c"} Jul 18 16:41:21.150364 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:21.150339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"e4006c4e1a54233e7238c285613bb876e0b7055e681dea9000d033182044544f"} Jul 18 16:41:22.160526 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.160465 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qmdfl" event={"ID":"64bc4b28-2f6c-4b23-93e2-baab4394473c","Type":"ContainerStarted","Data":"6c1b5972aec9fb035119713018ae14d92c32a3323d2b34a9df35d91a303004d6"} Jul 18 16:41:22.160965 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.160656 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:22.160965 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.160689 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qmdfl" event={"ID":"64bc4b28-2f6c-4b23-93e2-baab4394473c","Type":"ContainerStarted","Data":"3bb532c0c204010387e781023ac88c94404a6507cb957638c6b97fd05d05b054"} Jul 18 16:41:22.163027 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.163003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlbmx" event={"ID":"03f0e8a5-651a-4464-9dd4-f725eda6bd5c","Type":"ContainerStarted","Data":"e7533079f12c0d5adc5af57ff8480ce15730a6a418b58e9f443094f51f5e319b"} Jul 18 16:41:22.163154 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.163050 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlbmx" event={"ID":"03f0e8a5-651a-4464-9dd4-f725eda6bd5c","Type":"ContainerStarted","Data":"d571fb6f66eeb7550a658da9a56b401d78abe2dc43fda9249c78ecdaffd60994"} Jul 18 16:41:22.164642 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.164614 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0" exitCode=0 Jul 18 16:41:22.164762 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.164696 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0"} Jul 18 16:41:22.179443 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.179394 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qmdfl" podStartSLOduration=251.272124191 podStartE2EDuration="4m13.179375579s" totalImagesPullingTime="1.907251388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:41:22.178692303 +0000 UTC m=+287.317454965" watchObservedRunningTime="2026-07-18 16:41:22.179375579 +0000 UTC m=+287.318138243" Jul 18 16:41:22.195489 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:22.195444 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xlbmx" podStartSLOduration=2.924939702 podStartE2EDuration="4.195429376s" totalImagesPullingTime="1.193299796s" totalInitContainerRuntime="77.189878ms" isStatefulPod=true podCreationTimestamp="2026-07-18 16:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:41:22.194222632 +0000 UTC m=+287.332985296" watchObservedRunningTime="2026-07-18 16:41:22.195429376 +0000 UTC m=+287.334192039" Jul 18 16:41:24.175010 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.174974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6"} Jul 18 16:41:24.175436 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.175053 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d"} Jul 18 16:41:24.175436 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.175092 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928"} Jul 18 16:41:24.175436 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.175173 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809"} Jul 18 16:41:24.175436 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.175211 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986"} Jul 18 16:41:24.176759 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.176737 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"0bc40249336357ade088a2fb84eae65bf40dc21325e4fdc4e9af0352a205257a"} Jul 18 16:41:24.176819 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.176796 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"efcfbf14865fb2f9f2734abb98e87744fc30dd22c964ef06f9b839ca3348377e"} Jul 18 16:41:24.176859 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:24.176825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"1b18ccf2b434f0ddeb2155b5a6b4e89af4a29eeeea7ba531ad999ff6d864645e"} Jul 18 16:41:25.183035 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.182993 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerStarted","Data":"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5"} Jul 18 16:41:25.185770 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.185744 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"078723ee44757170d79bb08ee666e23893ca8babd054d263e2ffae07124f5feb"} Jul 18 16:41:25.185888 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.185807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"1d9d1a16afd679c8331110e66973d27414c3cd0d5b2b08d563c452657ce59745"} Jul 18 16:41:25.185888 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.185850 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:25.186018 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.185887 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" event={"ID":"a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417","Type":"ContainerStarted","Data":"0679fadb247304cfb846eaaf57d665fc86278a482baf5e578ec128928e51cd0f"} Jul 18 16:41:25.208958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.208914 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.912448595 podStartE2EDuration="6.208898491s" totalImagesPullingTime="3.271351425s" totalInitContainerRuntime="25.098471ms" isStatefulPod=false podCreationTimestamp="2026-07-18 16:41:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:41:25.208140667 +0000 UTC m=+290.346903329" watchObservedRunningTime="2026-07-18 16:41:25.208898491 +0000 UTC m=+290.347661154" Jul 18 16:41:25.232783 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:25.231457 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" podStartSLOduration=1.7603059760000002 podStartE2EDuration="5.231438549s" totalImagesPullingTime="3.471132573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:41:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:41:25.228808426 +0000 UTC m=+290.367571090" watchObservedRunningTime="2026-07-18 16:41:25.231438549 +0000 UTC m=+290.370201214" Jul 18 16:41:29.275456 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:41:29.275403 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gzzf2" podUID="63a253e8-58a9-4e0d-a5a6-3fea89b67a56" Jul 18 16:41:31.195626 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:31.195599 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7954b47d79-tl6wz" Jul 18 16:41:32.171697 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:32.171660 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qmdfl" Jul 18 16:41:35.211450 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:35.211422 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:41:35.211909 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:35.211742 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:41:35.218148 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:35.218127 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 16:41:42.274376 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:41:42.274337 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:42:38.566300 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566266 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:38.566796 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566576 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="alertmanager" containerID="cri-o://eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" gracePeriod=120 Jul 18 16:42:38.566796 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566627 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-metric" containerID="cri-o://9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" gracePeriod=120 Jul 18 16:42:38.566796 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566663 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-web" containerID="cri-o://a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" gracePeriod=120 Jul 18 16:42:38.566796 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566682 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy" containerID="cri-o://ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" gracePeriod=120 Jul 18 16:42:38.566796 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566756 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="prom-label-proxy" containerID="cri-o://579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" gracePeriod=120 Jul 18 16:42:38.567054 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:38.566783 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="config-reloader" containerID="cri-o://31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" gracePeriod=120 Jul 18 16:42:39.447081 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447035 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" exitCode=0 Jul 18 16:42:39.447081 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447067 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" exitCode=0 Jul 18 16:42:39.447081 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447074 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" exitCode=0 Jul 18 16:42:39.447081 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447080 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" exitCode=0 Jul 18 16:42:39.447395 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5"} Jul 18 16:42:39.447395 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447188 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d"} Jul 18 16:42:39.447395 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809"} Jul 18 16:42:39.447395 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.447263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986"} Jul 18 16:42:39.798843 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.798818 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:39.923958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.923915 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.923958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.923959 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924183 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.923992 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924183 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924008 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924183 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924139 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924339 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924185 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924339 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924212 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924339 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924244 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924339 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924290 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55rz8\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924338 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924349 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 16:42:39.924564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924368 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924412 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924453 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") pod \"b867f2d8-deb4-41bd-89af-62d82eac68ca\" (UID: \"b867f2d8-deb4-41bd-89af-62d82eac68ca\") " Jul 18 16:42:39.924814 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.924679 2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-metrics-client-ca\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:39.925080 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.925046 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 16:42:39.925623 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.925574 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 16:42:39.926984 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.926933 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume" (OuterVolumeSpecName: "config-volume") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.926984 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.926941 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 16:42:39.927147 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.927023 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.927147 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.927035 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.927147 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.927098 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.927656 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.927631 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out" (OuterVolumeSpecName: "config-out") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 16:42:39.927887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.927851 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.928532 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.928501 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8" (OuterVolumeSpecName: "kube-api-access-55rz8") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "kube-api-access-55rz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 16:42:39.931352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.931329 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:39.937828 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:39.937805 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config" (OuterVolumeSpecName: "web-config") pod "b867f2d8-deb4-41bd-89af-62d82eac68ca" (UID: "b867f2d8-deb4-41bd-89af-62d82eac68ca"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025087 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55rz8\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-kube-api-access-55rz8\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025117 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-main-tls\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025127 2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-cluster-tls-config\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025136 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-main-db\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025144 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b867f2d8-deb4-41bd-89af-62d82eac68ca-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025153 2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-volume\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025162 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025167 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025170 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b867f2d8-deb4-41bd-89af-62d82eac68ca-config-out\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025186 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b867f2d8-deb4-41bd-89af-62d82eac68ca-tls-assets\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025194 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025202 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-web-config\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.025525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.025210 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b867f2d8-deb4-41bd-89af-62d82eac68ca-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:42:40.453940 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.453908 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" exitCode=0 Jul 18 16:42:40.453940 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.453933 2567 generic.go:356] "Generic (PLEG): container finished" podID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerID="a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" exitCode=0 Jul 18 16:42:40.454125 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.453969 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6"} Jul 18 16:42:40.454125 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.453979 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.454125 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.454024 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928"} Jul 18 16:42:40.454125 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.454049 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b867f2d8-deb4-41bd-89af-62d82eac68ca","Type":"ContainerDied","Data":"e4006c4e1a54233e7238c285613bb876e0b7055e681dea9000d033182044544f"} Jul 18 16:42:40.454125 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.454093 2567 scope.go:117] "RemoveContainer" containerID="579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" Jul 18 16:42:40.461749 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.461656 2567 scope.go:117] "RemoveContainer" containerID="9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" Jul 18 16:42:40.468421 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.468402 2567 scope.go:117] "RemoveContainer" containerID="ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" Jul 18 16:42:40.476078 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.476061 2567 scope.go:117] "RemoveContainer" containerID="a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" Jul 18 16:42:40.476285 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.476268 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:40.481731 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.481709 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:40.482793 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.482778 2567 scope.go:117] "RemoveContainer" containerID="31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" Jul 18 16:42:40.489168 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.489148 2567 scope.go:117] "RemoveContainer" containerID="eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" Jul 18 16:42:40.496171 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.496147 2567 scope.go:117] "RemoveContainer" containerID="bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0" Jul 18 16:42:40.502626 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.502611 2567 scope.go:117] "RemoveContainer" containerID="579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" Jul 18 16:42:40.502894 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.502876 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5\": container with ID starting with 579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5 not found: ID does not exist" containerID="579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" Jul 18 16:42:40.502935 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.502904 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5"} err="failed to get container status \"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5\": rpc error: code = NotFound desc = could not find container \"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5\": container with ID starting with 579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5 not found: ID does not exist" Jul 18 16:42:40.502935 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.502921 2567 scope.go:117] "RemoveContainer" containerID="9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" Jul 18 16:42:40.503165 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.503149 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6\": container with ID starting with 9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6 not found: ID does not exist" containerID="9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" Jul 18 16:42:40.503214 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503186 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6"} err="failed to get container status \"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6\": rpc error: code = NotFound desc = could not find container \"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6\": container with ID starting with 9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6 not found: ID does not exist" Jul 18 16:42:40.503214 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503202 2567 scope.go:117] "RemoveContainer" containerID="ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" Jul 18 16:42:40.503404 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.503387 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d\": container with ID starting with ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d not found: ID does not exist" containerID="ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" Jul 18 16:42:40.503449 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503409 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d"} err="failed to get container status \"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d\": rpc error: code = NotFound desc = could not find container \"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d\": container with ID starting with ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d not found: ID does not exist" Jul 18 16:42:40.503449 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503426 2567 scope.go:117] "RemoveContainer" containerID="a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" Jul 18 16:42:40.503664 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.503635 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928\": container with ID starting with a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928 not found: ID does not exist" containerID="a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" Jul 18 16:42:40.503664 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503654 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928"} err="failed to get container status \"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928\": rpc error: code = NotFound desc = could not find container \"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928\": container with ID starting with a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928 not found: ID does not exist" Jul 18 16:42:40.503814 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503669 2567 scope.go:117] "RemoveContainer" containerID="31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" Jul 18 16:42:40.503895 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.503879 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809\": container with ID starting with 31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809 not found: ID does not exist" containerID="31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" Jul 18 16:42:40.503953 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503902 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809"} err="failed to get container status \"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809\": rpc error: code = NotFound desc = could not find container \"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809\": container with ID starting with 31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809 not found: ID does not exist" Jul 18 16:42:40.503953 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.503922 2567 scope.go:117] "RemoveContainer" containerID="eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" Jul 18 16:42:40.504153 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.504137 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986\": container with ID starting with eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986 not found: ID does not exist" containerID="eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" Jul 18 16:42:40.504190 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504158 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986"} err="failed to get container status \"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986\": rpc error: code = NotFound desc = could not find container \"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986\": container with ID starting with eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986 not found: ID does not exist" Jul 18 16:42:40.504190 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504177 2567 scope.go:117] "RemoveContainer" containerID="bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0" Jul 18 16:42:40.504396 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:42:40.504378 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0\": container with ID starting with bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0 not found: ID does not exist" containerID="bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0" Jul 18 16:42:40.504457 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504403 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0"} err="failed to get container status \"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0\": rpc error: code = NotFound desc = could not find container \"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0\": container with ID starting with bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0 not found: ID does not exist" Jul 18 16:42:40.504457 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504423 2567 scope.go:117] "RemoveContainer" containerID="579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5" Jul 18 16:42:40.504653 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504636 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5"} err="failed to get container status \"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5\": rpc error: code = NotFound desc = could not find container \"579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5\": container with ID starting with 579ecbafa8a716948239d4c9afadcfa0da2247949ac1ed9fff5d3432e91ad4d5 not found: ID does not exist" Jul 18 16:42:40.504724 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504654 2567 scope.go:117] "RemoveContainer" containerID="9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6" Jul 18 16:42:40.504880 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504862 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6"} err="failed to get container status \"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6\": rpc error: code = NotFound desc = could not find container \"9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6\": container with ID starting with 9ef20d6edced36e8318ee9f81e7a9dd8ed6f8bdede0224620841fd70aea25ea6 not found: ID does not exist" Jul 18 16:42:40.504924 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.504881 2567 scope.go:117] "RemoveContainer" containerID="ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d" Jul 18 16:42:40.505073 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505054 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d"} err="failed to get container status \"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d\": rpc error: code = NotFound desc = could not find container \"ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d\": container with ID starting with ff911a48f02f7d85d0b0a53f6e47080aea8bb5ee9742bbaf4b1e112027100e3d not found: ID does not exist" Jul 18 16:42:40.505138 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505074 2567 scope.go:117] "RemoveContainer" containerID="a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928" Jul 18 16:42:40.505276 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505261 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928"} err="failed to get container status \"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928\": rpc error: code = NotFound desc = could not find container \"a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928\": container with ID starting with a377376d2df22e494aa1a16994d43ee3dac9257e65e50e415c8b328c9c62a928 not found: ID does not exist" Jul 18 16:42:40.505326 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505277 2567 scope.go:117] "RemoveContainer" containerID="31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809" Jul 18 16:42:40.505464 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505446 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809"} err="failed to get container status \"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809\": rpc error: code = NotFound desc = could not find container \"31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809\": container with ID starting with 31ffef8234635c8bb34ad1140f5996c77851e6609e1c7ba1d25e0baa7fa1e809 not found: ID does not exist" Jul 18 16:42:40.505542 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505465 2567 scope.go:117] "RemoveContainer" containerID="eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986" Jul 18 16:42:40.505677 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505659 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986"} err="failed to get container status \"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986\": rpc error: code = NotFound desc = could not find container \"eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986\": container with ID starting with eafb078bb966ef301625d17edb6316204e142f30083f12330c92644a333a6986 not found: ID does not exist" Jul 18 16:42:40.505718 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505677 2567 scope.go:117] "RemoveContainer" containerID="bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0" Jul 18 16:42:40.505861 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.505846 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0"} err="failed to get container status \"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0\": rpc error: code = NotFound desc = could not find container \"bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0\": container with ID starting with bcc7b86dbf2195fe51f679f953dbac2b3e68fe9f6042582db76e16c015cd82a0 not found: ID does not exist" Jul 18 16:42:40.507794 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.507775 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:40.508006 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.507995 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="init-config-reloader" Jul 18 16:42:40.508041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508008 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="init-config-reloader" Jul 18 16:42:40.508041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508019 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="prom-label-proxy" Jul 18 16:42:40.508041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508024 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="prom-label-proxy" Jul 18 16:42:40.508041 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508039 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508045 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508051 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-web" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508056 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-web" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508062 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-metric" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508067 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-metric" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508073 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="config-reloader" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508077 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="config-reloader" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508087 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="alertmanager" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508091 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="alertmanager" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508126 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-metric" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508132 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="prom-label-proxy" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508139 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy-web" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508145 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="alertmanager" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508150 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="kube-rbac-proxy" Jul 18 16:42:40.508164 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.508156 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" containerName="config-reloader" Jul 18 16:42:40.512661 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.512647 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.515071 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515052 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 18 16:42:40.515188 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515123 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 18 16:42:40.515188 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515137 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 18 16:42:40.515336 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515185 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 18 16:42:40.515396 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515384 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 18 16:42:40.515492 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515416 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 18 16:42:40.515707 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515677 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 18 16:42:40.515852 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515828 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p9fdm\"" Jul 18 16:42:40.515974 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.515957 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 18 16:42:40.520613 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.520595 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 18 16:42:40.527204 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.527180 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:40.628383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628335 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628383 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628387 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628461 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628504 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-config-out\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628558 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-web-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628974 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.628974 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.628727 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfcqq\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-kube-api-access-wfcqq\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.729997 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.729784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733280 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733217 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733280 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733475 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.730266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733475 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733475 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733475 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-config-out\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733553 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733587 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733631 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733698 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733928 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733711 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-web-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733928 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.733928 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.733803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfcqq\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-kube-api-access-wfcqq\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.736234 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.736205 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.736429 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.736381 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b635e28-c576-4148-857f-08c7c68283c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.737099 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.736591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738160 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738096 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738346 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738273 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-web-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738346 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738295 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738346 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b635e28-c576-4148-857f-08c7c68283c8-config-out\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738591 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738573 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.738818 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.738801 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7b635e28-c576-4148-857f-08c7c68283c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.739043 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.739027 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.742039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.742021 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfcqq\" (UniqueName: \"kubernetes.io/projected/7b635e28-c576-4148-857f-08c7c68283c8-kube-api-access-wfcqq\") pod \"alertmanager-main-0\" (UID: \"7b635e28-c576-4148-857f-08c7c68283c8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.823016 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.822976 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 16:42:40.956256 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:40.956102 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 16:42:40.958669 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:42:40.958647 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b635e28_c576_4148_857f_08c7c68283c8.slice/crio-5f51c550116f3a1facb67d2a7ceddda592fb21c513c620283095187fb417dd7c WatchSource:0}: Error finding container 5f51c550116f3a1facb67d2a7ceddda592fb21c513c620283095187fb417dd7c: Status 404 returned error can't find the container with id 5f51c550116f3a1facb67d2a7ceddda592fb21c513c620283095187fb417dd7c Jul 18 16:42:41.278293 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:41.278213 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b867f2d8-deb4-41bd-89af-62d82eac68ca" path="/var/lib/kubelet/pods/b867f2d8-deb4-41bd-89af-62d82eac68ca/volumes" Jul 18 16:42:41.459528 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:41.459471 2567 generic.go:356] "Generic (PLEG): container finished" podID="7b635e28-c576-4148-857f-08c7c68283c8" containerID="dd36aff682f8825cdf729018d74b62e910316083f43e1ed9588ae978fd85eb73" exitCode=0 Jul 18 16:42:41.459678 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:41.459559 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerDied","Data":"dd36aff682f8825cdf729018d74b62e910316083f43e1ed9588ae978fd85eb73"} Jul 18 16:42:41.459678 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:41.459619 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"5f51c550116f3a1facb67d2a7ceddda592fb21c513c620283095187fb417dd7c"} Jul 18 16:42:42.467887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.467856 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"6ccbadc5de842fa29ee58a9b38fde83c64864d4b0fc3f6e93e4fac44369479bd"} Jul 18 16:42:42.468279 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.467920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"cd5a624edf705c23831815fa771d576fac1a661b257a1b252417f0f1df92d532"} Jul 18 16:42:42.468279 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.467947 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"ed9861af532c1450bfcce68884de6b3695e8fe8c264e0cb0eec33df65f22d3e2"} Jul 18 16:42:42.468279 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.467972 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"918997acd8dd7402269faf24052e89d5dd3ddc52b19ce5fb57a8cff39060131c"} Jul 18 16:42:42.468279 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.467996 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"f41fe33d6dc152bb47efe8f9f2ea4bcfb771a1963e5a31ab02206202e73607be"} Jul 18 16:42:42.468279 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.468019 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7b635e28-c576-4148-857f-08c7c68283c8","Type":"ContainerStarted","Data":"30c993c44d15a0a2273a7d10ef012208ed53857fba66c4025e2ad1108a055347"} Jul 18 16:42:42.492748 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.492694 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.47777103 podStartE2EDuration="2.49267682s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.90579ms" isStatefulPod=false podCreationTimestamp="2026-07-18 16:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:42:42.491855058 +0000 UTC m=+367.630617732" watchObservedRunningTime="2026-07-18 16:42:42.49267682 +0000 UTC m=+367.631439482" Jul 18 16:42:42.589735 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.589702 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cc76df989-n6rv8"] Jul 18 16:42:42.592888 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.592871 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.594973 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.594952 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2chvn\"" Jul 18 16:42:42.595091 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.594974 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 18 16:42:42.595208 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.595194 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 18 16:42:42.595277 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.595221 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 18 16:42:42.595332 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.595221 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 18 16:42:42.595438 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.595423 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 18 16:42:42.599549 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.599529 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 18 16:42:42.605847 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.605824 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cc76df989-n6rv8"] Jul 18 16:42:42.751792 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751691 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-federate-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.751792 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751736 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-metrics-client-ca\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.751792 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751764 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.751792 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.752076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751830 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.752076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmxl5\" (UniqueName: \"kubernetes.io/projected/dc046995-5a1e-4777-9233-b7c413319c52-kube-api-access-hmxl5\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.752076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.752076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.751964 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-serving-certs-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.852996 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.852957 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.852996 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.852999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-serving-certs-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-federate-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853074 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-metrics-client-ca\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853377 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853377 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853377 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmxl5\" (UniqueName: \"kubernetes.io/projected/dc046995-5a1e-4777-9233-b7c413319c52-kube-api-access-hmxl5\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.853883 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-serving-certs-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.854004 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.853968 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.854221 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.854190 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc046995-5a1e-4777-9233-b7c413319c52-metrics-client-ca\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.855645 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.855617 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-federate-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.855847 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.855822 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.855971 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.855956 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.856116 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.856094 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc046995-5a1e-4777-9233-b7c413319c52-telemeter-client-tls\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.861389 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.861367 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmxl5\" (UniqueName: \"kubernetes.io/projected/dc046995-5a1e-4777-9233-b7c413319c52-kube-api-access-hmxl5\") pod \"telemeter-client-6cc76df989-n6rv8\" (UID: \"dc046995-5a1e-4777-9233-b7c413319c52\") " pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:42.902490 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:42.902464 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" Jul 18 16:42:43.028963 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:43.028768 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cc76df989-n6rv8"] Jul 18 16:42:43.032905 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:42:43.032878 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc046995_5a1e_4777_9233_b7c413319c52.slice/crio-cdc7e17995ade8b5cd1944115d24943fcd4207f95a2ec5f31cb9c7f087043df2 WatchSource:0}: Error finding container cdc7e17995ade8b5cd1944115d24943fcd4207f95a2ec5f31cb9c7f087043df2: Status 404 returned error can't find the container with id cdc7e17995ade8b5cd1944115d24943fcd4207f95a2ec5f31cb9c7f087043df2 Jul 18 16:42:43.034879 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:43.034862 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 16:42:43.473241 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:43.473205 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" event={"ID":"dc046995-5a1e-4777-9233-b7c413319c52","Type":"ContainerStarted","Data":"cdc7e17995ade8b5cd1944115d24943fcd4207f95a2ec5f31cb9c7f087043df2"} Jul 18 16:42:45.482373 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:45.482339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" event={"ID":"dc046995-5a1e-4777-9233-b7c413319c52","Type":"ContainerStarted","Data":"a48031c1fcd304d4123e0a1c22c3aaa8e166eefe21d51131d6954fcb8250c5c4"} Jul 18 16:42:45.482728 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:45.482393 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" event={"ID":"dc046995-5a1e-4777-9233-b7c413319c52","Type":"ContainerStarted","Data":"e7e02659087ca42b820ef814b6e9726d2c32d311accdbc4767522250477437b3"} Jul 18 16:42:45.482728 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:45.482413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" event={"ID":"dc046995-5a1e-4777-9233-b7c413319c52","Type":"ContainerStarted","Data":"7d0dc806878e12c0f7e12ba977cfa535945ba6f7faf2c5b1e972bec38cff59c3"} Jul 18 16:42:45.504245 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:45.504206 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cc76df989-n6rv8" podStartSLOduration=1.6413372750000002 podStartE2EDuration="3.504190938s" totalImagesPullingTime="1.862853663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:42:45.503749528 +0000 UTC m=+370.642512202" watchObservedRunningTime="2026-07-18 16:42:45.504190938 +0000 UTC m=+370.642953601" Jul 18 16:42:49.205632 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:49.205590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:42:49.207966 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:49.207941 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a253e8-58a9-4e0d-a5a6-3fea89b67a56-metrics-certs\") pod \"network-metrics-daemon-gzzf2\" (UID: \"63a253e8-58a9-4e0d-a5a6-3fea89b67a56\") " pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:42:49.478074 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:49.477991 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pm4pn\"" Jul 18 16:42:49.485760 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:49.485742 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gzzf2" Jul 18 16:42:49.606437 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:49.606390 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gzzf2"] Jul 18 16:42:49.608903 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:42:49.608881 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a253e8_58a9_4e0d_a5a6_3fea89b67a56.slice/crio-4b01eae730eae2828c9de6437e527929e0b58ed7e9bab33e494f3622d4b45a3d WatchSource:0}: Error finding container 4b01eae730eae2828c9de6437e527929e0b58ed7e9bab33e494f3622d4b45a3d: Status 404 returned error can't find the container with id 4b01eae730eae2828c9de6437e527929e0b58ed7e9bab33e494f3622d4b45a3d Jul 18 16:42:50.504579 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:50.504545 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gzzf2" event={"ID":"63a253e8-58a9-4e0d-a5a6-3fea89b67a56","Type":"ContainerStarted","Data":"4b01eae730eae2828c9de6437e527929e0b58ed7e9bab33e494f3622d4b45a3d"} Jul 18 16:42:51.509816 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:51.509782 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gzzf2" event={"ID":"63a253e8-58a9-4e0d-a5a6-3fea89b67a56","Type":"ContainerStarted","Data":"4c75d84b42dada05545fe4bcedcdffd2f5013f8104cbeb8e63f23b22b074d591"} Jul 18 16:42:51.510227 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:51.509838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gzzf2" event={"ID":"63a253e8-58a9-4e0d-a5a6-3fea89b67a56","Type":"ContainerStarted","Data":"7f7fcfea54cd5fbb39244fb752505f9d4b38179446e8bde7f730e82a728b8422"} Jul 18 16:42:51.528269 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:42:51.528227 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gzzf2" podStartSLOduration=375.708136726 podStartE2EDuration="6m16.528212345s" totalImagesPullingTime="820.075619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:42:51.52719881 +0000 UTC m=+376.665961471" watchObservedRunningTime="2026-07-18 16:42:51.528212345 +0000 UTC m=+376.666975008" Jul 18 16:45:48.543253 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.543220 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb"] Jul 18 16:45:48.546251 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.546235 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.549297 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.549277 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 16:45:48.549620 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.549604 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7t2k9\"" Jul 18 16:45:48.550483 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.550466 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 16:45:48.571637 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.571615 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb"] Jul 18 16:45:48.732905 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.732869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swkvc\" (UniqueName: \"kubernetes.io/projected/44ba7eee-05d9-45e2-82fc-e742620b09a7-kube-api-access-swkvc\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.732905 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.732909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44ba7eee-05d9-45e2-82fc-e742620b09a7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.833782 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.833706 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swkvc\" (UniqueName: \"kubernetes.io/projected/44ba7eee-05d9-45e2-82fc-e742620b09a7-kube-api-access-swkvc\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.833782 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.833753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44ba7eee-05d9-45e2-82fc-e742620b09a7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.834137 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.834119 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44ba7eee-05d9-45e2-82fc-e742620b09a7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.841525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.841477 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swkvc\" (UniqueName: \"kubernetes.io/projected/44ba7eee-05d9-45e2-82fc-e742620b09a7-kube-api-access-swkvc\") pod \"openshift-lws-operator-fd8ccff4c-qw5jb\" (UID: \"44ba7eee-05d9-45e2-82fc-e742620b09a7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.855332 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.855307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" Jul 18 16:45:48.979607 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:48.979422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb"] Jul 18 16:45:48.982173 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:45:48.982132 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44ba7eee_05d9_45e2_82fc_e742620b09a7.slice/crio-92e00531b2d9399350840d73b62a7f0fe0292631ad75e37d337f6129e4b4ac35 WatchSource:0}: Error finding container 92e00531b2d9399350840d73b62a7f0fe0292631ad75e37d337f6129e4b4ac35: Status 404 returned error can't find the container with id 92e00531b2d9399350840d73b62a7f0fe0292631ad75e37d337f6129e4b4ac35 Jul 18 16:45:49.155178 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:49.155140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" event={"ID":"44ba7eee-05d9-45e2-82fc-e742620b09a7","Type":"ContainerStarted","Data":"92e00531b2d9399350840d73b62a7f0fe0292631ad75e37d337f6129e4b4ac35"} Jul 18 16:45:52.173680 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:52.173645 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" event={"ID":"44ba7eee-05d9-45e2-82fc-e742620b09a7","Type":"ContainerStarted","Data":"77d849e51605a29eb8d1274a3b541ef7beee51e27cb65d810f09a4a8f54aced9"} Jul 18 16:45:52.189190 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:45:52.189128 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qw5jb" podStartSLOduration=1.840289662 podStartE2EDuration="4.189084883s" totalImagesPullingTime="2.348795221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:45:52.187969572 +0000 UTC m=+557.326732235" watchObservedRunningTime="2026-07-18 16:45:52.189084883 +0000 UTC m=+557.327847545" Jul 18 16:46:08.640440 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.640402 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw"] Jul 18 16:46:08.643594 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.643576 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.646096 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.646070 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kffsd\"" Jul 18 16:46:08.646248 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.646140 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 16:46:08.646597 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.646550 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 16:46:08.646665 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.646587 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 16:46:08.646665 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.646586 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 16:46:08.661923 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.661897 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw"] Jul 18 16:46:08.670011 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.669988 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-manifests\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.670127 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.670070 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-tmp\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.670127 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.670104 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.670232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.670138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-webhook-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.670232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.670164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2c99\" (UniqueName: \"kubernetes.io/projected/08e06ba3-2865-4762-a85e-c2f0906bed81-kube-api-access-j2c99\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.770702 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.770669 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-tmp\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.770702 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.770713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.770964 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.770740 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-webhook-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.770964 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.770759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2c99\" (UniqueName: \"kubernetes.io/projected/08e06ba3-2865-4762-a85e-c2f0906bed81-kube-api-access-j2c99\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.770964 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.770794 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-manifests\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.771201 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.771174 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-manifests\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.773076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.773051 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08e06ba3-2865-4762-a85e-c2f0906bed81-tmp\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.773256 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.773234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-webhook-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.773323 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.773305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08e06ba3-2865-4762-a85e-c2f0906bed81-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.788450 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.788426 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2c99\" (UniqueName: \"kubernetes.io/projected/08e06ba3-2865-4762-a85e-c2f0906bed81-kube-api-access-j2c99\") pod \"opendatahub-operator-controller-manager-c768ddb5d-wp5gw\" (UID: \"08e06ba3-2865-4762-a85e-c2f0906bed81\") " pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:08.953723 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:08.953634 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:09.084949 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:09.084880 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw"] Jul 18 16:46:09.088766 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:09.088707 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e06ba3_2865_4762_a85e_c2f0906bed81.slice/crio-4621aa07ba9eecf83d6e5500e3ccc985ebda6d032cdd00564eced6f4d40c842f WatchSource:0}: Error finding container 4621aa07ba9eecf83d6e5500e3ccc985ebda6d032cdd00564eced6f4d40c842f: Status 404 returned error can't find the container with id 4621aa07ba9eecf83d6e5500e3ccc985ebda6d032cdd00564eced6f4d40c842f Jul 18 16:46:09.240768 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:09.240681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" event={"ID":"08e06ba3-2865-4762-a85e-c2f0906bed81","Type":"ContainerStarted","Data":"4621aa07ba9eecf83d6e5500e3ccc985ebda6d032cdd00564eced6f4d40c842f"} Jul 18 16:46:13.258730 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:13.258694 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" event={"ID":"08e06ba3-2865-4762-a85e-c2f0906bed81","Type":"ContainerStarted","Data":"a4984a2975794dd2f9bc403d4279d5ac7cbd1facbbb8c59fa38aac13e20bc9ad"} Jul 18 16:46:14.264145 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:14.264107 2567 generic.go:356] "Generic (PLEG): container finished" podID="08e06ba3-2865-4762-a85e-c2f0906bed81" containerID="a4984a2975794dd2f9bc403d4279d5ac7cbd1facbbb8c59fa38aac13e20bc9ad" exitCode=0 Jul 18 16:46:14.264539 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:14.264161 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" event={"ID":"08e06ba3-2865-4762-a85e-c2f0906bed81","Type":"ContainerDied","Data":"a4984a2975794dd2f9bc403d4279d5ac7cbd1facbbb8c59fa38aac13e20bc9ad"} Jul 18 16:46:15.269681 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:15.269647 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" event={"ID":"08e06ba3-2865-4762-a85e-c2f0906bed81","Type":"ContainerStarted","Data":"283b5780448a4cb4d68ae29850cc0f84ed7c1dcd0453e301f409b37d351a7bc4"} Jul 18 16:46:15.270064 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:15.269754 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:15.291573 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:15.291500 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" podStartSLOduration=2.66388787 podStartE2EDuration="7.291486116s" totalImagesPullingTime="3.547959129s" totalInitContainerRuntime="1.079639117s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:15.290030336 +0000 UTC m=+580.428793002" watchObservedRunningTime="2026-07-18 16:46:15.291486116 +0000 UTC m=+580.430248779" Jul 18 16:46:26.275225 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:26.275195 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c768ddb5d-wp5gw" Jul 18 16:46:28.303887 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.303850 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz"] Jul 18 16:46:28.320225 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.320199 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.323268 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.323245 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 16:46:28.323469 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.323453 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 16:46:28.324289 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.324271 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 18 16:46:28.325377 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.325359 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 18 16:46:28.325651 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.325531 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jwf5n\"" Jul 18 16:46:28.329872 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.329853 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz"] Jul 18 16:46:28.434775 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.434744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932ff878-af0b-4930-b433-b066253f1e87-tls-certs\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.434945 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.434798 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klx2x\" (UniqueName: \"kubernetes.io/projected/932ff878-af0b-4930-b433-b066253f1e87-kube-api-access-klx2x\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.434945 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.434896 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/932ff878-af0b-4930-b433-b066253f1e87-tmp\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.535456 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.535424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klx2x\" (UniqueName: \"kubernetes.io/projected/932ff878-af0b-4930-b433-b066253f1e87-kube-api-access-klx2x\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.535636 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.535474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/932ff878-af0b-4930-b433-b066253f1e87-tmp\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.535636 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.535615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932ff878-af0b-4930-b433-b066253f1e87-tls-certs\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.537781 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.537759 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/932ff878-af0b-4930-b433-b066253f1e87-tmp\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.537968 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.537948 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932ff878-af0b-4930-b433-b066253f1e87-tls-certs\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.562432 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.562375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klx2x\" (UniqueName: \"kubernetes.io/projected/932ff878-af0b-4930-b433-b066253f1e87-kube-api-access-klx2x\") pod \"kube-auth-proxy-6c475ff8c-bv4fz\" (UID: \"932ff878-af0b-4930-b433-b066253f1e87\") " pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.630326 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.630297 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" Jul 18 16:46:28.778669 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:28.778631 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz"] Jul 18 16:46:28.780718 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:28.780685 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932ff878_af0b_4930_b433_b066253f1e87.slice/crio-3bec1483008071a6831fd3328798fc57a87ba937dba63684a778401119087341 WatchSource:0}: Error finding container 3bec1483008071a6831fd3328798fc57a87ba937dba63684a778401119087341: Status 404 returned error can't find the container with id 3bec1483008071a6831fd3328798fc57a87ba937dba63684a778401119087341 Jul 18 16:46:29.324005 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.323967 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" event={"ID":"932ff878-af0b-4930-b433-b066253f1e87","Type":"ContainerStarted","Data":"3bec1483008071a6831fd3328798fc57a87ba937dba63684a778401119087341"} Jul 18 16:46:29.661621 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.661585 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-697w9"] Jul 18 16:46:29.664971 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.664952 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:29.667557 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.667536 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 18 16:46:29.675981 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.675956 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-697w9"] Jul 18 16:46:29.745501 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.745467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4np2\" (UniqueName: \"kubernetes.io/projected/805b14f7-698f-43a7-8ef3-31b15d16dc3d-kube-api-access-w4np2\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:29.745674 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.745566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:29.846833 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.846793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:29.846991 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.846878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4np2\" (UniqueName: \"kubernetes.io/projected/805b14f7-698f-43a7-8ef3-31b15d16dc3d-kube-api-access-w4np2\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:29.846991 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:46:29.846948 2567 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 18 16:46:29.847088 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:46:29.847023 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs podName:805b14f7-698f-43a7-8ef3-31b15d16dc3d nodeName:}" failed. No retries permitted until 2026-07-18 16:46:30.347003896 +0000 UTC m=+595.485766558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs") pod "model-serving-api-54d89bd5fc-697w9" (UID: "805b14f7-698f-43a7-8ef3-31b15d16dc3d") : secret "model-serving-api-tls" not found Jul 18 16:46:29.857524 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:29.857491 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4np2\" (UniqueName: \"kubernetes.io/projected/805b14f7-698f-43a7-8ef3-31b15d16dc3d-kube-api-access-w4np2\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:30.351342 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:30.351312 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:30.353757 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:30.353740 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805b14f7-698f-43a7-8ef3-31b15d16dc3d-tls-certs\") pod \"model-serving-api-54d89bd5fc-697w9\" (UID: \"805b14f7-698f-43a7-8ef3-31b15d16dc3d\") " pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:30.578212 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:30.578173 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:30.718337 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:30.718309 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-697w9"] Jul 18 16:46:30.720560 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:30.720529 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805b14f7_698f_43a7_8ef3_31b15d16dc3d.slice/crio-cd383ab28dd2898cdc298115e1d3f2c907e37684af17a14bda736f95644a2615 WatchSource:0}: Error finding container cd383ab28dd2898cdc298115e1d3f2c907e37684af17a14bda736f95644a2615: Status 404 returned error can't find the container with id cd383ab28dd2898cdc298115e1d3f2c907e37684af17a14bda736f95644a2615 Jul 18 16:46:31.333816 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:31.333779 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" event={"ID":"805b14f7-698f-43a7-8ef3-31b15d16dc3d","Type":"ContainerStarted","Data":"cd383ab28dd2898cdc298115e1d3f2c907e37684af17a14bda736f95644a2615"} Jul 18 16:46:32.339559 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:32.339502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" event={"ID":"932ff878-af0b-4930-b433-b066253f1e87","Type":"ContainerStarted","Data":"a494162c40cc125354bcdd79cb01230f20e2ba0a24f47a7d5da4c27032fdef60"} Jul 18 16:46:32.355827 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:32.355768 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c475ff8c-bv4fz" podStartSLOduration=1.527806799 podStartE2EDuration="4.355752323s" totalImagesPullingTime="2.827945524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:32.355281166 +0000 UTC m=+597.494043828" watchObservedRunningTime="2026-07-18 16:46:32.355752323 +0000 UTC m=+597.494514989" Jul 18 16:46:33.344870 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:33.344825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" event={"ID":"805b14f7-698f-43a7-8ef3-31b15d16dc3d","Type":"ContainerStarted","Data":"8dbe3417808b03f4e6d46a7b9ca9487887c714dbf7aa307e33adcbc686bd4385"} Jul 18 16:46:33.345277 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:33.344991 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:33.359764 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:33.359716 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" podStartSLOduration=2.251727419 podStartE2EDuration="4.359700523s" totalImagesPullingTime="2.107973104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:33.358910049 +0000 UTC m=+598.497672748" watchObservedRunningTime="2026-07-18 16:46:33.359700523 +0000 UTC m=+598.498463186" Jul 18 16:46:35.155898 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.155863 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86"] Jul 18 16:46:35.159183 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.159165 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.161643 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.161622 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 16:46:35.161751 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.161623 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-djz9f\"" Jul 18 16:46:35.171293 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.171267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86"] Jul 18 16:46:35.243338 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.243312 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:46:35.243595 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.243582 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:46:35.298131 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298278 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298278 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298170 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298278 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298248 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298380 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298303 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298380 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298324 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298380 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298380 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8xl9\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-kube-api-access-g8xl9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.298498 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.298387 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399543 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399499 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399543 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399544 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399781 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399562 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399781 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8xl9\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-kube-api-access-g8xl9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399781 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399708 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399941 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399787 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399941 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399835 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399941 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.399941 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.399922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.400139 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.400018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.400139 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.400064 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.400245 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.400191 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.400304 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.400280 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.401922 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.401899 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.402133 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.402115 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.403442 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.403423 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 16:46:35.411237 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.411190 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.420447 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.420423 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.420576 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.420553 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8xl9\" (UniqueName: \"kubernetes.io/projected/50c0faeb-95d1-489b-b792-e67b37b5d0c6-kube-api-access-g8xl9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rpn86\" (UID: \"50c0faeb-95d1-489b-b792-e67b37b5d0c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.470935 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.470904 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-djz9f\"" Jul 18 16:46:35.478683 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.478658 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:46:35.616902 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:35.616880 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86"] Jul 18 16:46:35.618785 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:35.618759 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c0faeb_95d1_489b_b792_e67b37b5d0c6.slice/crio-3b53fbbcbb67d76e91f49b734bae7dd4fd0888b4f11796a0fbd21e2761e06f3c WatchSource:0}: Error finding container 3b53fbbcbb67d76e91f49b734bae7dd4fd0888b4f11796a0fbd21e2761e06f3c: Status 404 returned error can't find the container with id 3b53fbbcbb67d76e91f49b734bae7dd4fd0888b4f11796a0fbd21e2761e06f3c Jul 18 16:46:36.357958 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.357925 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" event={"ID":"50c0faeb-95d1-489b-b792-e67b37b5d0c6","Type":"ContainerStarted","Data":"3b53fbbcbb67d76e91f49b734bae7dd4fd0888b4f11796a0fbd21e2761e06f3c"} Jul 18 16:46:36.975579 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.975543 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6"] Jul 18 16:46:36.978648 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.978631 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:36.983001 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.982978 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 18 16:46:36.983982 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.983963 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 18 16:46:36.984077 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.983985 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-64997\"" Jul 18 16:46:36.985664 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.985646 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 18 16:46:36.996397 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:36.996376 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6"] Jul 18 16:46:37.114604 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114576 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114776 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114612 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114776 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114643 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs49v\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-kube-api-access-qs49v\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114776 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114698 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114776 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114776 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1e349e-3ecd-46d1-beda-6c7a76d82894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.114960 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.114797 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.215936 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.215902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.215936 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.215938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216203 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.215965 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs49v\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-kube-api-access-qs49v\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216203 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.215985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216203 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.216015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216203 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.216031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1e349e-3ecd-46d1-beda-6c7a76d82894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216203 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.216050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.216780 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.216753 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.218564 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.218537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.218684 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.218587 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.218744 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.218708 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1e349e-3ecd-46d1-beda-6c7a76d82894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.218799 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.218766 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.224227 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.224179 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.224358 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.224339 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs49v\" (UniqueName: \"kubernetes.io/projected/8d1e349e-3ecd-46d1-beda-6c7a76d82894-kube-api-access-qs49v\") pod \"istiod-openshift-gateway-94bb8fbfd-6whd6\" (UID: \"8d1e349e-3ecd-46d1-beda-6c7a76d82894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.287600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.287533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:37.420050 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:37.420017 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6"] Jul 18 16:46:37.423141 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:37.423111 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1e349e_3ecd_46d1_beda_6c7a76d82894.slice/crio-4cfde0f00ce7addbd6b667af7fabe464c30e01706ef4711da5e0598ec464c2a0 WatchSource:0}: Error finding container 4cfde0f00ce7addbd6b667af7fabe464c30e01706ef4711da5e0598ec464c2a0: Status 404 returned error can't find the container with id 4cfde0f00ce7addbd6b667af7fabe464c30e01706ef4711da5e0598ec464c2a0 Jul 18 16:46:38.368484 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.368443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" event={"ID":"8d1e349e-3ecd-46d1-beda-6c7a76d82894","Type":"ContainerStarted","Data":"4cfde0f00ce7addbd6b667af7fabe464c30e01706ef4711da5e0598ec464c2a0"} Jul 18 16:46:38.933328 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.933293 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4"] Jul 18 16:46:38.936801 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.936771 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:38.937065 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.937041 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z"] Jul 18 16:46:38.939341 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.939288 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 18 16:46:38.940067 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.940048 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hvht6\"" Jul 18 16:46:38.940627 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.940608 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:38.942798 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.942780 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 18 16:46:38.942951 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.942816 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gf8tw\"" Jul 18 16:46:38.950630 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.950567 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4"] Jul 18 16:46:38.954282 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:38.954261 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z"] Jul 18 16:46:39.034031 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.033998 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfj7\" (UniqueName: \"kubernetes.io/projected/aeb58dd5-641b-4d4b-9da9-52f4f543d061-kube-api-access-qzfj7\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.034195 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.034038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p58lk\" (UniqueName: \"kubernetes.io/projected/cdd95720-ff0e-4bbb-86e3-18e356908c76-kube-api-access-p58lk\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.034195 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.034075 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdd95720-ff0e-4bbb-86e3-18e356908c76-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.034195 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.034098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.135531 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.135480 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p58lk\" (UniqueName: \"kubernetes.io/projected/cdd95720-ff0e-4bbb-86e3-18e356908c76-kube-api-access-p58lk\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.135695 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.135558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdd95720-ff0e-4bbb-86e3-18e356908c76-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.135695 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.135580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.135695 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.135662 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfj7\" (UniqueName: \"kubernetes.io/projected/aeb58dd5-641b-4d4b-9da9-52f4f543d061-kube-api-access-qzfj7\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.135879 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:46:39.135800 2567 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 18 16:46:39.135879 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:46:39.135867 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert podName:aeb58dd5-641b-4d4b-9da9-52f4f543d061 nodeName:}" failed. No retries permitted until 2026-07-18 16:46:39.635845738 +0000 UTC m=+604.774608391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert") pod "kserve-controller-manager-d6f4b65b8-ghnr4" (UID: "aeb58dd5-641b-4d4b-9da9-52f4f543d061") : secret "kserve-webhook-server-cert" not found Jul 18 16:46:39.138212 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.138192 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdd95720-ff0e-4bbb-86e3-18e356908c76-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.147968 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.147941 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p58lk\" (UniqueName: \"kubernetes.io/projected/cdd95720-ff0e-4bbb-86e3-18e356908c76-kube-api-access-p58lk\") pod \"llmisvc-controller-manager-777c6bd7d5-rjw9z\" (UID: \"cdd95720-ff0e-4bbb-86e3-18e356908c76\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.150609 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.150566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfj7\" (UniqueName: \"kubernetes.io/projected/aeb58dd5-641b-4d4b-9da9-52f4f543d061-kube-api-access-qzfj7\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.261938 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.261855 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:39.640070 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.640031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.642733 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.642707 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeb58dd5-641b-4d4b-9da9-52f4f543d061-cert\") pod \"kserve-controller-manager-d6f4b65b8-ghnr4\" (UID: \"aeb58dd5-641b-4d4b-9da9-52f4f543d061\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.855122 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.855090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:39.965325 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.965285 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:46:39.965642 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:39.965363 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:46:40.034583 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.034549 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z"] Jul 18 16:46:40.038102 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:40.038077 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcdd95720_ff0e_4bbb_86e3_18e356908c76.slice/crio-5dc192d006dbf93d21e20732603d27a01e16fdd53ada9556bb24f425bdc59beb WatchSource:0}: Error finding container 5dc192d006dbf93d21e20732603d27a01e16fdd53ada9556bb24f425bdc59beb: Status 404 returned error can't find the container with id 5dc192d006dbf93d21e20732603d27a01e16fdd53ada9556bb24f425bdc59beb Jul 18 16:46:40.064667 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.064645 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4"] Jul 18 16:46:40.067151 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:46:40.067122 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb58dd5_641b_4d4b_9da9_52f4f543d061.slice/crio-fc5050460d251dd083de024ed303cd97d29dd808d38787d76ae369417211698d WatchSource:0}: Error finding container fc5050460d251dd083de024ed303cd97d29dd808d38787d76ae369417211698d: Status 404 returned error can't find the container with id fc5050460d251dd083de024ed303cd97d29dd808d38787d76ae369417211698d Jul 18 16:46:40.379373 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.379338 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" event={"ID":"cdd95720-ff0e-4bbb-86e3-18e356908c76","Type":"ContainerStarted","Data":"5dc192d006dbf93d21e20732603d27a01e16fdd53ada9556bb24f425bdc59beb"} Jul 18 16:46:40.380398 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.380376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" event={"ID":"aeb58dd5-641b-4d4b-9da9-52f4f543d061","Type":"ContainerStarted","Data":"fc5050460d251dd083de024ed303cd97d29dd808d38787d76ae369417211698d"} Jul 18 16:46:40.381874 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.381851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" event={"ID":"8d1e349e-3ecd-46d1-beda-6c7a76d82894","Type":"ContainerStarted","Data":"072c014ef1e51c6a87d71e8716eb0bae3bc3f883794cd2ef6c76596fbc69d192"} Jul 18 16:46:40.381992 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.381979 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:46:40.407352 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:40.407300 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" podStartSLOduration=1.86739589 podStartE2EDuration="4.407287421s" totalImagesPullingTime="2.539891531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:40.405145737 +0000 UTC m=+605.543908435" watchObservedRunningTime="2026-07-18 16:46:40.407287421 +0000 UTC m=+605.546050084" Jul 18 16:46:41.390976 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:41.390936 2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6whd6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 16:46:41.391417 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:41.391073 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" podUID="8d1e349e-3ecd-46d1-beda-6c7a76d82894" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 16:46:42.395760 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:42.395723 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" event={"ID":"aeb58dd5-641b-4d4b-9da9-52f4f543d061","Type":"ContainerStarted","Data":"5a91e871e3d81eea87ec03447e3293d5b9b08016d96a4cb861dc59149e3bf816"} Jul 18 16:46:42.396169 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:42.395814 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:46:42.413879 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:42.413465 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" podStartSLOduration=2.211788106 podStartE2EDuration="4.41344385s" totalImagesPullingTime="2.201655744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:42.410213173 +0000 UTC m=+607.548975837" watchObservedRunningTime="2026-07-18 16:46:42.41344385 +0000 UTC m=+607.552206516" Jul 18 16:46:43.402269 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:43.402226 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" event={"ID":"cdd95720-ff0e-4bbb-86e3-18e356908c76","Type":"ContainerStarted","Data":"e7ca555cb564beedc50924c9ad2f5444b067c9cc7e47ca032135092ab88adab1"} Jul 18 16:46:43.402713 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:43.402297 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:46:43.418244 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:43.418192 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" podStartSLOduration=2.518848926 podStartE2EDuration="5.41816031s" totalImagesPullingTime="2.899311384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:46:43.418043117 +0000 UTC m=+608.556805800" watchObservedRunningTime="2026-07-18 16:46:43.41816031 +0000 UTC m=+608.556922972" Jul 18 16:46:44.352950 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:44.352916 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-697w9" Jul 18 16:46:44.390634 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:46:44.390605 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6whd6" Jul 18 16:47:13.407422 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:13.407345 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ghnr4" Jul 18 16:47:14.410160 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:14.410132 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rjw9z" Jul 18 16:47:26.562886 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:26.562848 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:47:26.563206 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:26.562933 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:47:26.563206 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:26.562967 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:47:27.589036 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:27.588998 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" event={"ID":"50c0faeb-95d1-489b-b792-e67b37b5d0c6","Type":"ContainerStarted","Data":"475a1e37d8935cfadb3c3064e4544aac01bcc3de0ec116f856bdeba8bbe305c2"} Jul 18 16:47:27.608065 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:27.607951 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" podStartSLOduration=1.666064715 podStartE2EDuration="52.607930422s" totalImagesPullingTime="50.941865707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:47:27.60632665 +0000 UTC m=+652.745089348" watchObservedRunningTime="2026-07-18 16:47:27.607930422 +0000 UTC m=+652.746693131" Jul 18 16:47:28.479641 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:28.479607 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:47:28.483897 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:28.483870 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:47:28.593283 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:28.593251 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:47:28.594117 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:47:28.594099 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rpn86" Jul 18 16:48:05.835841 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.835799 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97"] Jul 18 16:48:05.839608 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.839584 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:05.841931 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.841910 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 16:48:05.842821 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.842803 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-krtd7\"" Jul 18 16:48:05.842907 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.842803 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 18 16:48:05.842907 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.842851 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 16:48:05.849752 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.849726 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97"] Jul 18 16:48:05.989090 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:05.989050 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdfnz\" (UniqueName: \"kubernetes.io/projected/010700c4-200a-4c7b-abe9-247f22afeadd-kube-api-access-wdfnz\") pod \"dns-operator-controller-manager-864dd78c4c-4rl97\" (UID: \"010700c4-200a-4c7b-abe9-247f22afeadd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:06.090530 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.090412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdfnz\" (UniqueName: \"kubernetes.io/projected/010700c4-200a-4c7b-abe9-247f22afeadd-kube-api-access-wdfnz\") pod \"dns-operator-controller-manager-864dd78c4c-4rl97\" (UID: \"010700c4-200a-4c7b-abe9-247f22afeadd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:06.097927 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.097899 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdfnz\" (UniqueName: \"kubernetes.io/projected/010700c4-200a-4c7b-abe9-247f22afeadd-kube-api-access-wdfnz\") pod \"dns-operator-controller-manager-864dd78c4c-4rl97\" (UID: \"010700c4-200a-4c7b-abe9-247f22afeadd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:06.150650 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.150619 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:06.296222 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.296187 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97"] Jul 18 16:48:06.299232 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:48:06.299205 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010700c4_200a_4c7b_abe9_247f22afeadd.slice/crio-f7c2397bc00b4587f869a48e3a3bdc46d82d92610f67e1096d8bebac9ced2ecb WatchSource:0}: Error finding container f7c2397bc00b4587f869a48e3a3bdc46d82d92610f67e1096d8bebac9ced2ecb: Status 404 returned error can't find the container with id f7c2397bc00b4587f869a48e3a3bdc46d82d92610f67e1096d8bebac9ced2ecb Jul 18 16:48:06.301327 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.301308 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 16:48:06.755409 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:06.755376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" event={"ID":"010700c4-200a-4c7b-abe9-247f22afeadd","Type":"ContainerStarted","Data":"f7c2397bc00b4587f869a48e3a3bdc46d82d92610f67e1096d8bebac9ced2ecb"} Jul 18 16:48:08.768340 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:08.768300 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" event={"ID":"010700c4-200a-4c7b-abe9-247f22afeadd","Type":"ContainerStarted","Data":"1212cd088609fad188194bdfc5711fd07b7b2abb1f0657de824d2fa99a449622"} Jul 18 16:48:08.768734 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:08.768426 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:08.786771 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:08.786719 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" podStartSLOduration=2.081037928 podStartE2EDuration="3.786703785s" totalImagesPullingTime="1.705665857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:48:08.785083997 +0000 UTC m=+693.923846662" watchObservedRunningTime="2026-07-18 16:48:08.786703785 +0000 UTC m=+693.925466448" Jul 18 16:48:16.099892 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.099853 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84"] Jul 18 16:48:16.103456 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.103434 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.106051 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.106034 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bmv67\"" Jul 18 16:48:16.116462 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.116437 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84"] Jul 18 16:48:16.252304 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.252263 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84"] Jul 18 16:48:16.252567 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:16.252546 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-v69q8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" podUID="d86d797b-f9b1-4fad-84c2-ea03fe4962b5" Jul 18 16:48:16.264724 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.264692 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84"] Jul 18 16:48:16.282833 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.282797 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v69q8\" (UniqueName: \"kubernetes.io/projected/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-kube-api-access-v69q8\") pod \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.283007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.282866 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.301247 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.301218 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:16.304433 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.304418 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.329623 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.329587 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:16.384413 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.384378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v69q8\" (UniqueName: \"kubernetes.io/projected/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-kube-api-access-v69q8\") pod \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.384610 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.384435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.384858 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.384836 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.396577 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:16.396555 2567 projected.go:196] Error preparing data for projected volume kube-api-access-v69q8 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-rsv84" not found Jul 18 16:48:16.396717 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:16.396621 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-kube-api-access-v69q8 podName:d86d797b-f9b1-4fad-84c2-ea03fe4962b5 nodeName:}" failed. No retries permitted until 2026-07-18 16:48:16.896604717 +0000 UTC m=+702.035367358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v69q8" (UniqueName: "kubernetes.io/projected/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-kube-api-access-v69q8") pod "kuadrant-operator-controller-manager-55467d9f7-rsv84" (UID: "d86d797b-f9b1-4fad-84c2-ea03fe4962b5") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-rsv84" not found Jul 18 16:48:16.485983 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.485945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf7nf\" (UniqueName: \"kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.486173 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.486018 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.587127 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.587094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf7nf\" (UniqueName: \"kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.587294 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.587154 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.587532 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.587495 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.596349 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.596319 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf7nf\" (UniqueName: \"kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf\") pod \"kuadrant-operator-controller-manager-55467d9f7-fgjf6\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.614158 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.614118 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:16.745482 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.745449 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:16.749087 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:48:16.749058 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac45f4ae_3007_436e_97bf_95442656eed4.slice/crio-23f740209cf8766d98001e6417d2dbc4d9217fd0016e09ea671764fb6637e5f3 WatchSource:0}: Error finding container 23f740209cf8766d98001e6417d2dbc4d9217fd0016e09ea671764fb6637e5f3: Status 404 returned error can't find the container with id 23f740209cf8766d98001e6417d2dbc4d9217fd0016e09ea671764fb6637e5f3 Jul 18 16:48:16.803736 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.803710 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.803919 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.803738 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" event={"ID":"ac45f4ae-3007-436e-97bf-95442656eed4","Type":"ContainerStarted","Data":"23f740209cf8766d98001e6417d2dbc4d9217fd0016e09ea671764fb6637e5f3"} Jul 18 16:48:16.806065 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:16.806036 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" is forbidden: User \"system:node:ip-10-0-139-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-1.ec2.internal' and this object" podUID="d86d797b-f9b1-4fad-84c2-ea03fe4962b5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.811660 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.811642 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.813722 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:16.813700 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" is forbidden: User \"system:node:ip-10-0-139-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-1.ec2.internal' and this object" podUID="d86d797b-f9b1-4fad-84c2-ea03fe4962b5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:16.991412 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.991328 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume\") pod \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\" (UID: \"d86d797b-f9b1-4fad-84c2-ea03fe4962b5\") " Jul 18 16:48:16.991591 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.991566 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v69q8\" (UniqueName: \"kubernetes.io/projected/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-kube-api-access-v69q8\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:48:16.991651 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:16.991634 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d86d797b-f9b1-4fad-84c2-ea03fe4962b5" (UID: "d86d797b-f9b1-4fad-84c2-ea03fe4962b5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 16:48:17.092919 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:17.092883 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d86d797b-f9b1-4fad-84c2-ea03fe4962b5-extensions-socket-volume\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:48:17.281310 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:17.281237 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d86d797b-f9b1-4fad-84c2-ea03fe4962b5" path="/var/lib/kubelet/pods/d86d797b-f9b1-4fad-84c2-ea03fe4962b5/volumes" Jul 18 16:48:17.808191 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:17.808154 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:17.813828 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:17.813793 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rsv84\" is forbidden: User \"system:node:ip-10-0-139-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-1.ec2.internal' and this object" podUID="d86d797b-f9b1-4fad-84c2-ea03fe4962b5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rsv84" Jul 18 16:48:19.775957 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:19.775920 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4rl97" Jul 18 16:48:21.829712 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:21.829677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" event={"ID":"ac45f4ae-3007-436e-97bf-95442656eed4","Type":"ContainerStarted","Data":"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c"} Jul 18 16:48:21.830147 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:21.829866 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:21.850873 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:21.850825 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" podStartSLOduration=1.6833442280000002 podStartE2EDuration="5.85080652s" totalImagesPullingTime="4.167462292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:48:21.849694138 +0000 UTC m=+706.988456800" watchObservedRunningTime="2026-07-18 16:48:21.85080652 +0000 UTC m=+706.989569184" Jul 18 16:48:32.395365 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.395283 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:32.395852 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.395570 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" podUID="ac45f4ae-3007-436e-97bf-95442656eed4" containerName="manager" containerID="cri-o://c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c" gracePeriod=10 Jul 18 16:48:32.397104 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.397084 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:32.647409 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.647351 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:32.731903 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.731871 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume\") pod \"ac45f4ae-3007-436e-97bf-95442656eed4\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " Jul 18 16:48:32.732101 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.731954 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf7nf\" (UniqueName: \"kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf\") pod \"ac45f4ae-3007-436e-97bf-95442656eed4\" (UID: \"ac45f4ae-3007-436e-97bf-95442656eed4\") " Jul 18 16:48:32.732301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.732277 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ac45f4ae-3007-436e-97bf-95442656eed4" (UID: "ac45f4ae-3007-436e-97bf-95442656eed4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 16:48:32.734076 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.734051 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf" (OuterVolumeSpecName: "kube-api-access-sf7nf") pod "ac45f4ae-3007-436e-97bf-95442656eed4" (UID: "ac45f4ae-3007-436e-97bf-95442656eed4"). InnerVolumeSpecName "kube-api-access-sf7nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 16:48:32.832676 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.832635 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf7nf\" (UniqueName: \"kubernetes.io/projected/ac45f4ae-3007-436e-97bf-95442656eed4-kube-api-access-sf7nf\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:48:32.832676 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.832668 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac45f4ae-3007-436e-97bf-95442656eed4-extensions-socket-volume\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 16:48:32.881481 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.881448 2567 generic.go:356] "Generic (PLEG): container finished" podID="ac45f4ae-3007-436e-97bf-95442656eed4" containerID="c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c" exitCode=0 Jul 18 16:48:32.881667 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.881490 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" Jul 18 16:48:32.881667 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.881499 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" event={"ID":"ac45f4ae-3007-436e-97bf-95442656eed4","Type":"ContainerDied","Data":"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c"} Jul 18 16:48:32.881667 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.881576 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6" event={"ID":"ac45f4ae-3007-436e-97bf-95442656eed4","Type":"ContainerDied","Data":"23f740209cf8766d98001e6417d2dbc4d9217fd0016e09ea671764fb6637e5f3"} Jul 18 16:48:32.881667 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.881605 2567 scope.go:117] "RemoveContainer" containerID="c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c" Jul 18 16:48:32.890365 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.890348 2567 scope.go:117] "RemoveContainer" containerID="c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c" Jul 18 16:48:32.890631 ip-10-0-139-1 kubenswrapper[2567]: E0718 16:48:32.890609 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c\": container with ID starting with c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c not found: ID does not exist" containerID="c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c" Jul 18 16:48:32.890691 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.890638 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c"} err="failed to get container status \"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c\": rpc error: code = NotFound desc = could not find container \"c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c\": container with ID starting with c1f9e56022d875d2de4c16e7a45339c6264fa97e63f365df6c0796aee3c7749c not found: ID does not exist" Jul 18 16:48:32.902200 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.902147 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:32.910293 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:32.910270 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fgjf6"] Jul 18 16:48:33.280734 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:33.280657 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac45f4ae-3007-436e-97bf-95442656eed4" path="/var/lib/kubelet/pods/ac45f4ae-3007-436e-97bf-95442656eed4/volumes" Jul 18 16:48:48.695705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.695668 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s"] Jul 18 16:48:48.696194 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.696032 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac45f4ae-3007-436e-97bf-95442656eed4" containerName="manager" Jul 18 16:48:48.696194 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.696045 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac45f4ae-3007-436e-97bf-95442656eed4" containerName="manager" Jul 18 16:48:48.696194 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.696104 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac45f4ae-3007-436e-97bf-95442656eed4" containerName="manager" Jul 18 16:48:48.699769 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.699750 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.702083 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.702060 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h8kk8\"" Jul 18 16:48:48.710446 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.710423 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s"] Jul 18 16:48:48.765007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.764973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58629c39-717d-4869-b1d2-b21f641450f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-645bc\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-kube-api-access-645bc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765063 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58629c39-717d-4869-b1d2-b21f641450f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765219 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.765486 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.765293 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866322 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866322 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866351 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58629c39-717d-4869-b1d2-b21f641450f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866600 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58629c39-717d-4869-b1d2-b21f641450f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866904 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-645bc\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-kube-api-access-645bc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866904 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866832 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.866904 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.866866 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.867113 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.867091 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.867160 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.867119 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.867416 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.867399 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58629c39-717d-4869-b1d2-b21f641450f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.868723 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.868706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58629c39-717d-4869-b1d2-b21f641450f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.869196 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.869166 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58629c39-717d-4869-b1d2-b21f641450f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.874232 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.874207 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:48.874332 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:48.874270 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-645bc\" (UniqueName: \"kubernetes.io/projected/58629c39-717d-4869-b1d2-b21f641450f7-kube-api-access-645bc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-f6v4s\" (UID: \"58629c39-717d-4869-b1d2-b21f641450f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:49.010318 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.010233 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:49.146752 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.146727 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s"] Jul 18 16:48:49.149137 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:48:49.149110 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58629c39_717d_4869_b1d2_b21f641450f7.slice/crio-d246a9d109ac478e915a1627c9f6cab885efb73fb1b35dd2e5a416984480c314 WatchSource:0}: Error finding container d246a9d109ac478e915a1627c9f6cab885efb73fb1b35dd2e5a416984480c314: Status 404 returned error can't find the container with id d246a9d109ac478e915a1627c9f6cab885efb73fb1b35dd2e5a416984480c314 Jul 18 16:48:49.151337 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.151304 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:48:49.151410 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.151394 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:48:49.151492 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.151458 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:48:49.959174 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.959138 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" event={"ID":"58629c39-717d-4869-b1d2-b21f641450f7","Type":"ContainerStarted","Data":"1b8ba61cedf53b6b1b1876b8b30328bf5a420ab01bf833de8425f11b661751e7"} Jul 18 16:48:49.959569 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.959191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" event={"ID":"58629c39-717d-4869-b1d2-b21f641450f7","Type":"ContainerStarted","Data":"d246a9d109ac478e915a1627c9f6cab885efb73fb1b35dd2e5a416984480c314"} Jul 18 16:48:49.981150 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:49.981100 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" podStartSLOduration=1.981086299 podStartE2EDuration="1.981086299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:48:49.978795846 +0000 UTC m=+735.117558529" watchObservedRunningTime="2026-07-18 16:48:49.981086299 +0000 UTC m=+735.119849015" Jul 18 16:48:50.011050 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:50.011019 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:50.015634 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:50.015612 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:50.963897 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:50.963863 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:50.964790 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:50.964770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:48:51.438413 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.438384 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:48:51.441889 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.441867 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.444198 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.444177 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 18 16:48:51.444305 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.444179 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bnb7h\"" Jul 18 16:48:51.448785 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.448762 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:48:51.478365 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.478332 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:48:51.592478 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.592443 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77c17dfc-d6e9-4680-892b-e0fb9075bd89-config-file\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.592687 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.592585 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvwgs\" (UniqueName: \"kubernetes.io/projected/77c17dfc-d6e9-4680-892b-e0fb9075bd89-kube-api-access-hvwgs\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.693737 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.693630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77c17dfc-d6e9-4680-892b-e0fb9075bd89-config-file\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.693737 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.693738 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvwgs\" (UniqueName: \"kubernetes.io/projected/77c17dfc-d6e9-4680-892b-e0fb9075bd89-kube-api-access-hvwgs\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.694308 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.694285 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77c17dfc-d6e9-4680-892b-e0fb9075bd89-config-file\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.702642 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.702622 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvwgs\" (UniqueName: \"kubernetes.io/projected/77c17dfc-d6e9-4680-892b-e0fb9075bd89-kube-api-access-hvwgs\") pod \"limitador-limitador-54d6d988d5-jvx76\" (UID: \"77c17dfc-d6e9-4680-892b-e0fb9075bd89\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.753579 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.753551 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:51.876922 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.876899 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:48:51.878943 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:48:51.878919 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c17dfc_d6e9_4680_892b_e0fb9075bd89.slice/crio-fcbf8b1caa48f777681a42f6a0156a727caec882bb5bf0103eee74a60e2800cb WatchSource:0}: Error finding container fcbf8b1caa48f777681a42f6a0156a727caec882bb5bf0103eee74a60e2800cb: Status 404 returned error can't find the container with id fcbf8b1caa48f777681a42f6a0156a727caec882bb5bf0103eee74a60e2800cb Jul 18 16:48:51.969031 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:51.968945 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" event={"ID":"77c17dfc-d6e9-4680-892b-e0fb9075bd89","Type":"ContainerStarted","Data":"fcbf8b1caa48f777681a42f6a0156a727caec882bb5bf0103eee74a60e2800cb"} Jul 18 16:48:54.992139 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:54.992047 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" event={"ID":"77c17dfc-d6e9-4680-892b-e0fb9075bd89","Type":"ContainerStarted","Data":"6f5dab0c324edc44040f1ad0f9c826740c5e91473ee8ea2c0cba08220fe992e1"} Jul 18 16:48:54.992540 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:54.992204 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:48:55.009792 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:48:55.009744 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" podStartSLOduration=1.182995894 podStartE2EDuration="4.009732063s" totalImagesPullingTime="2.826736169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:48:55.007228833 +0000 UTC m=+740.145991496" watchObservedRunningTime="2026-07-18 16:48:55.009732063 +0000 UTC m=+740.148494726" Jul 18 16:49:05.996800 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:49:05.996772 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jvx76" Jul 18 16:50:01.973530 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:01.973424 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:50:10.335987 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.335956 2567 generic.go:356] "Generic (PLEG): container finished" podID="58629c39-717d-4869-b1d2-b21f641450f7" containerID="1b8ba61cedf53b6b1b1876b8b30328bf5a420ab01bf833de8425f11b661751e7" exitCode=0 Jul 18 16:50:10.336490 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.336035 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" event={"ID":"58629c39-717d-4869-b1d2-b21f641450f7","Type":"ContainerDied","Data":"1b8ba61cedf53b6b1b1876b8b30328bf5a420ab01bf833de8425f11b661751e7"} Jul 18 16:50:10.340525 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.336971 2567 scope.go:117] "RemoveContainer" containerID="1b8ba61cedf53b6b1b1876b8b30328bf5a420ab01bf833de8425f11b661751e7" Jul 18 16:50:10.345552 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.345500 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:50:10.345649 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.345624 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:50:10.345691 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:10.345665 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 16:50:11.341885 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:11.341855 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" event={"ID":"58629c39-717d-4869-b1d2-b21f641450f7","Type":"ContainerStarted","Data":"8491a797bac2c7ec523b24813311605e3f55aa6b3b90748dd010384649a4b591"} Jul 18 16:50:12.011301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:12.011259 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:50:12.011534 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:12.011390 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-f6v4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 18 16:50:12.011534 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:12.011434 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" podUID="58629c39-717d-4869-b1d2-b21f641450f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 18 16:50:13.010800 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:13.010762 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-f6v4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 18 16:50:13.011385 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:13.011345 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" podUID="58629c39-717d-4869-b1d2-b21f641450f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 18 16:50:14.010883 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:14.010843 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-f6v4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 18 16:50:14.011487 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:14.011451 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" podUID="58629c39-717d-4869-b1d2-b21f641450f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 18 16:50:15.011378 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:15.011339 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-f6v4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 18 16:50:15.011997 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:15.011962 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" podUID="58629c39-717d-4869-b1d2-b21f641450f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 18 16:50:16.013861 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:16.013829 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:50:16.014231 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:16.014091 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:50:16.014811 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:16.014793 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-f6v4s" Jul 18 16:50:16.913249 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:16.913215 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:50:31.069196 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.069164 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x"] Jul 18 16:50:31.072539 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.072499 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.076186 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.076156 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 16:50:31.076301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.076153 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9d62d\"" Jul 18 16:50:31.076301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.076165 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 18 16:50:31.076301 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.076281 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 16:50:31.083772 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.083752 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x"] Jul 18 16:50:31.171863 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.171825 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.172028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.171885 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.172028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.171958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.172028 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.171986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9wg9\" (UniqueName: \"kubernetes.io/projected/0f3828c8-4171-45cc-85a4-b2be0a46411a-kube-api-access-z9wg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.172159 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.172033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.172159 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.172110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f3828c8-4171-45cc-85a4-b2be0a46411a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.272877 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.272838 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.272890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.272916 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9wg9\" (UniqueName: \"kubernetes.io/projected/0f3828c8-4171-45cc-85a4-b2be0a46411a-kube-api-access-z9wg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.272972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273039 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.273019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f3828c8-4171-45cc-85a4-b2be0a46411a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273236 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.273065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273392 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.273366 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273392 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.273384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.273529 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.273405 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.275224 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.275202 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f3828c8-4171-45cc-85a4-b2be0a46411a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.275353 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.275332 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f3828c8-4171-45cc-85a4-b2be0a46411a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.281865 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.281841 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9wg9\" (UniqueName: \"kubernetes.io/projected/0f3828c8-4171-45cc-85a4-b2be0a46411a-kube-api-access-z9wg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qb98x\" (UID: \"0f3828c8-4171-45cc-85a4-b2be0a46411a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.384030 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.383998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:31.511255 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.511233 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x"] Jul 18 16:50:31.513034 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:50:31.512996 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3828c8_4171_45cc_85a4_b2be0a46411a.slice/crio-366f03ea12c373675ba512f389b73fcf5a44117451db8cb2b4a8fa49a9724020 WatchSource:0}: Error finding container 366f03ea12c373675ba512f389b73fcf5a44117451db8cb2b4a8fa49a9724020: Status 404 returned error can't find the container with id 366f03ea12c373675ba512f389b73fcf5a44117451db8cb2b4a8fa49a9724020 Jul 18 16:50:31.729613 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:31.729424 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:50:32.437472 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:32.437425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" event={"ID":"0f3828c8-4171-45cc-85a4-b2be0a46411a","Type":"ContainerStarted","Data":"366f03ea12c373675ba512f389b73fcf5a44117451db8cb2b4a8fa49a9724020"} Jul 18 16:50:33.443767 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:33.443732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" event={"ID":"0f3828c8-4171-45cc-85a4-b2be0a46411a","Type":"ContainerStarted","Data":"793fa87bf3262e77e183aa465718d0f04b2999c2908a48b30437b622d62525af"} Jul 18 16:50:33.466545 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:33.466479 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" podStartSLOduration=1.4668145 podStartE2EDuration="2.466464385s" totalImagesPullingTime="999.649885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:50:33.464839025 +0000 UTC m=+838.603601689" watchObservedRunningTime="2026-07-18 16:50:33.466464385 +0000 UTC m=+838.605227048" Jul 18 16:50:41.385138 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:41.385097 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:41.385570 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:41.385164 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:41.397967 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:41.397941 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:41.489734 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:41.489702 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qb98x" Jul 18 16:50:46.423628 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:46.423598 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:50:58.920918 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:50:58.920884 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:51:08.444471 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.444438 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm"] Jul 18 16:51:08.453943 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.453912 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.456589 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.456565 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 18 16:51:08.457214 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.457191 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm"] Jul 18 16:51:08.511719 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511685 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5ad940d-06f2-4794-b277-df8728231542-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.511719 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.511915 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.511915 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511811 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.511915 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4xhd\" (UniqueName: \"kubernetes.io/projected/d5ad940d-06f2-4794-b277-df8728231542-kube-api-access-b4xhd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.511915 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.511862 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613016 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.612981 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613016 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613018 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4xhd\" (UniqueName: \"kubernetes.io/projected/d5ad940d-06f2-4794-b277-df8728231542-kube-api-access-b4xhd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613241 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613241 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613078 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5ad940d-06f2-4794-b277-df8728231542-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613241 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613241 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613522 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613654 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613552 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.613705 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.613656 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.615409 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.615378 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5ad940d-06f2-4794-b277-df8728231542-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.615724 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.615706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5ad940d-06f2-4794-b277-df8728231542-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.621842 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.621817 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4xhd\" (UniqueName: \"kubernetes.io/projected/d5ad940d-06f2-4794-b277-df8728231542-kube-api-access-b4xhd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rscjm\" (UID: \"d5ad940d-06f2-4794-b277-df8728231542\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.765671 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.765586 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:08.897612 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:08.897577 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm"] Jul 18 16:51:08.899863 ip-10-0-139-1 kubenswrapper[2567]: W0718 16:51:08.899827 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ad940d_06f2_4794_b277_df8728231542.slice/crio-0f82fd40f638d6866a2049bbde16e5e81854993eab647cb7b872c7df1d13b018 WatchSource:0}: Error finding container 0f82fd40f638d6866a2049bbde16e5e81854993eab647cb7b872c7df1d13b018: Status 404 returned error can't find the container with id 0f82fd40f638d6866a2049bbde16e5e81854993eab647cb7b872c7df1d13b018 Jul 18 16:51:09.615007 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:09.614973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" event={"ID":"d5ad940d-06f2-4794-b277-df8728231542","Type":"ContainerStarted","Data":"37670df8b714f5c24bcfd44278f750123520bbf8ef0fcc6194f183d78d7dfb3f"} Jul 18 16:51:09.615403 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:09.615028 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" event={"ID":"d5ad940d-06f2-4794-b277-df8728231542","Type":"ContainerStarted","Data":"0f82fd40f638d6866a2049bbde16e5e81854993eab647cb7b872c7df1d13b018"} Jul 18 16:51:09.630981 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:09.630938 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" podStartSLOduration=1.462315276 podStartE2EDuration="1.630922727s" totalImagesPullingTime="168.607451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 16:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 16:51:09.629386908 +0000 UTC m=+874.768149571" watchObservedRunningTime="2026-07-18 16:51:09.630922727 +0000 UTC m=+874.769685389" Jul 18 16:51:10.521016 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:10.520980 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:51:18.766061 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:18.766027 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:18.766467 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:18.766293 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:18.778748 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:18.778726 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:19.677216 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:19.677185 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rscjm" Jul 18 16:51:31.732585 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:31.732490 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:51:35.272224 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:35.272196 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:51:35.273980 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:51:35.273950 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:53:08.417025 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:53:08.416941 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:53:17.734576 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:53:17.734541 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:53:28.451198 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:53:28.451162 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:55:11.424794 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:55:11.424755 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:55:47.017386 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:55:47.017353 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:55:56.524710 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:55:56.524670 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:07.113403 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:07.113314 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:15.710414 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:15.710378 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:28.123431 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:28.123390 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:35.310128 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:35.310098 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:56:35.312875 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:35.312849 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 16:56:37.922486 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:37.922450 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:49.326487 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:49.326458 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:56:59.048924 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:56:59.048887 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:57:21.221108 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:57:21.221072 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:57:33.019439 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:57:33.019360 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:57:56.621395 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:57:56.621356 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:58:07.323271 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:58:07.323237 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:58:21.920971 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:58:21.920935 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:58:32.225884 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:58:32.223174 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:58:41.519617 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:58:41.519587 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:58:51.013282 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:58:51.013245 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 16:59:00.232415 ip-10-0-139-1 kubenswrapper[2567]: I0718 16:59:00.232316 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:00:49.839612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:00:49.839580 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:00:59.550991 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:00:59.550955 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:01:10.249620 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:10.249586 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:01:22.544466 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:22.544428 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:01:32.144762 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:32.144726 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:01:35.339679 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:35.339645 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:01:35.342968 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:35.342947 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:01:42.782959 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:42.782921 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:01:51.240308 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:01:51.240270 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:02:02.440593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:02:02.440500 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:02:13.937040 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:02:13.937006 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:02:22.748711 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:02:22.748677 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:02:33.737071 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:02:33.737033 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:02:54.446751 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:02:54.446708 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:03:04.038547 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:03:04.038496 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:03:14.740525 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:03:14.740475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:03:26.048947 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:03:26.048906 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:04:01.848433 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:04:01.848397 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:04:15.643093 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:04:15.643053 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:04:22.741322 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:04:22.741284 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:04:39.150613 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:04:39.150571 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:04:48.744248 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:04:48.744213 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:00.939026 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:00.938923 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:11.550105 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:11.550054 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:20.650968 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:20.650931 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:29.742314 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:29.742281 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:43.143317 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:43.143284 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:05:52.143334 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:05:52.143290 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:01.742445 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:01.742407 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:12.840781 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:12.840750 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:24.246358 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:24.246308 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:33.543263 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:33.543228 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:35.368752 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:35.368724 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:06:35.372395 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:35.372373 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:06:45.242912 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:45.242876 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:06:56.943259 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:06:56.943227 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:06.141999 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:06.141963 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:19.442745 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:19.442706 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:28.643526 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:28.643483 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:38.541309 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:38.541277 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:39.255817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.255781 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:07:39.259559 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.259537 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.262306 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.262278 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e2278917-openshift-default-dockercfg-5hd5q\"" Jul 18 17:07:39.282214 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.282185 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:07:39.401091 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401057 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401091 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401092 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401307 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401307 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401170 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401307 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401203 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2rs\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401307 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401262 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401494 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401494 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.401494 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.401389 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502316 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502280 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502316 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502319 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502342 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502418 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2rs\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502612 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502542 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.502962 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.502902 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.503053 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.503031 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.503118 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.503079 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.503161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.503122 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.503398 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.503374 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.504847 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.504827 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.505078 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.505054 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.510672 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.510604 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.511019 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.510999 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2rs\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs\") pod \"e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.570348 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.570308 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:39.713578 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.713553 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:07:39.715770 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:07:39.715738 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54116eb9_b883_49de_8211_129a69c1d7f9.slice/crio-1c2c1e8aa0ff432c65ffeab4cdcfac1a59f884319afeab8e32353efe3d21efef WatchSource:0}: Error finding container 1c2c1e8aa0ff432c65ffeab4cdcfac1a59f884319afeab8e32353efe3d21efef: Status 404 returned error can't find the container with id 1c2c1e8aa0ff432c65ffeab4cdcfac1a59f884319afeab8e32353efe3d21efef Jul 18 17:07:39.718035 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.717993 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:39.718116 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.718090 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:39.718153 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:39.718134 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:40.250279 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:40.250247 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerStarted","Data":"fef73e95e2dcf721325810ae9779a6f8c360e4b581d2061037575914cb5aab7c"} Jul 18 17:07:40.250446 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:40.250295 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerStarted","Data":"1c2c1e8aa0ff432c65ffeab4cdcfac1a59f884319afeab8e32353efe3d21efef"} Jul 18 17:07:40.271481 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:40.271432 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podStartSLOduration=1.271418415 podStartE2EDuration="1.271418415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:07:40.2710942 +0000 UTC m=+1865.409856863" watchObservedRunningTime="2026-07-18 17:07:40.271418415 +0000 UTC m=+1865.410181079" Jul 18 17:07:40.570951 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:40.570868 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:40.575664 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:40.575640 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:41.254580 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:41.254545 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:41.255428 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:41.255405 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:46.639694 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:46.639661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:07:55.321818 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.321784 2567 generic.go:356] "Generic (PLEG): container finished" podID="54116eb9-b883-49de-8211-129a69c1d7f9" containerID="fef73e95e2dcf721325810ae9779a6f8c360e4b581d2061037575914cb5aab7c" exitCode=0 Jul 18 17:07:55.322230 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.321858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerDied","Data":"fef73e95e2dcf721325810ae9779a6f8c360e4b581d2061037575914cb5aab7c"} Jul 18 17:07:55.322276 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.322262 2567 scope.go:117] "RemoveContainer" containerID="fef73e95e2dcf721325810ae9779a6f8c360e4b581d2061037575914cb5aab7c" Jul 18 17:07:55.324120 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.323562 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:55.324120 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.323743 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:55.324120 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:55.323838 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:07:56.254611 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:56.254578 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:56.331601 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:56.331558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerStarted","Data":"78ab9673d044b952a09233bfc104b3e5b930f9ee1f6d73c2c8bd3d2380c7ce2f"} Jul 18 17:07:56.571102 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:56.571007 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:07:56.572265 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:56.572233 2567 patch_prober.go:28] interesting pod/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 18 17:07:56.572387 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:56.572293 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 18 17:07:57.571335 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:57.571245 2567 patch_prober.go:28] interesting pod/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 18 17:07:57.571335 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:57.571317 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 18 17:07:58.571559 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:58.571493 2567 patch_prober.go:28] interesting pod/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 18 17:07:58.572040 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:58.571599 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 18 17:07:59.570946 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:59.570907 2567 patch_prober.go:28] interesting pod/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 18 17:07:59.571249 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:07:59.570966 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 18 17:08:00.574584 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:00.574544 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:08:00.574953 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:00.574830 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:08:00.575460 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:00.575444 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:08:20.457639 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:20.457606 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:08:22.349442 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:22.349408 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:08:22.349934 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:22.349632 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" containerID="cri-o://78ab9673d044b952a09233bfc104b3e5b930f9ee1f6d73c2c8bd3d2380c7ce2f" gracePeriod=30 Jul 18 17:08:27.483953 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.483912 2567 generic.go:356] "Generic (PLEG): container finished" podID="54116eb9-b883-49de-8211-129a69c1d7f9" containerID="78ab9673d044b952a09233bfc104b3e5b930f9ee1f6d73c2c8bd3d2380c7ce2f" exitCode=0 Jul 18 17:08:27.484329 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.483976 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerDied","Data":"78ab9673d044b952a09233bfc104b3e5b930f9ee1f6d73c2c8bd3d2380c7ce2f"} Jul 18 17:08:27.484329 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.484035 2567 scope.go:117] "RemoveContainer" containerID="fef73e95e2dcf721325810ae9779a6f8c360e4b581d2061037575914cb5aab7c" Jul 18 17:08:27.606637 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.606613 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:08:27.740595 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740565 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740621 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740672 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp2rs\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740724 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740760 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740948 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740793 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740948 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740818 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740948 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740840 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.740948 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.740872 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy\") pod \"54116eb9-b883-49de-8211-129a69c1d7f9\" (UID: \"54116eb9-b883-49de-8211-129a69c1d7f9\") " Jul 18 17:08:27.741147 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741002 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:08:27.741207 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741171 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:08:27.741292 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741262 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:08:27.741349 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741338 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54116eb9-b883-49de-8211-129a69c1d7f9-istiod-ca-cert\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.741407 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741348 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:08:27.741407 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741357 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.741782 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.741755 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:08:27.743092 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.743066 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:08:27.743194 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.743149 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:08:27.743303 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.743275 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs" (OuterVolumeSpecName: "kube-api-access-lp2rs") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "kube-api-access-lp2rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:08:27.743392 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.743351 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54116eb9-b883-49de-8211-129a69c1d7f9" (UID: "54116eb9-b883-49de-8211-129a69c1d7f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 17:08:27.842809 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842778 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp2rs\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-kube-api-access-lp2rs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.842809 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842806 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-data\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.842809 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842815 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54116eb9-b883-49de-8211-129a69c1d7f9-istio-podinfo\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.843022 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842825 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-credential-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.843022 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842833 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-workload-certs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.843022 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842841 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54116eb9-b883-49de-8211-129a69c1d7f9-istio-envoy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:27.843022 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:27.842848 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54116eb9-b883-49de-8211-129a69c1d7f9-istio-token\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:08:28.489782 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:28.489752 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" Jul 18 17:08:28.490212 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:28.489800 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm" event={"ID":"54116eb9-b883-49de-8211-129a69c1d7f9","Type":"ContainerDied","Data":"1c2c1e8aa0ff432c65ffeab4cdcfac1a59f884319afeab8e32353efe3d21efef"} Jul 18 17:08:28.490212 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:28.489851 2567 scope.go:117] "RemoveContainer" containerID="78ab9673d044b952a09233bfc104b3e5b930f9ee1f6d73c2c8bd3d2380c7ce2f" Jul 18 17:08:28.525642 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:28.525613 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:08:28.527913 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:28.527882 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e2278917-openshift-default-5db94d477c-5fdjm"] Jul 18 17:08:29.280567 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:29.280502 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" path="/var/lib/kubelet/pods/54116eb9-b883-49de-8211-129a69c1d7f9/volumes" Jul 18 17:08:32.350216 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:08:32.350183 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:09:06.047859 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:06.047824 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:09:09.021619 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.021582 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:09.022065 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.021986 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:09:09.022065 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.022002 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:09:09.022178 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.022145 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:09:09.025138 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.025120 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.027758 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.027738 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b26f5ca7-openshift-default-dockercfg-phsjm\"" Jul 18 17:09:09.037931 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.037907 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:09.115786 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.115741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.115973 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.115790 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.115973 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.115826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.115973 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.115901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48p9c\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.116180 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.115992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.116180 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.116025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.116180 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.116055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.116180 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.116095 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.116180 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.116135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.217502 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.217458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.217533 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.217957 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218008 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218268 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218797 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218439 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218797 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218797 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.218797 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.218749 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48p9c\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.223118 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.219226 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.223118 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.219901 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.224976 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.224913 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.225189 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.225165 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.227765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.227746 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.228907 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.228884 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.229272 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.229254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48p9c\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c\") pod \"e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.335461 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.335374 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:09.474054 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.474024 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:09.474695 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:09:09.474659 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89ebe70_70e3_4c3a_9ec1_2f74e5de274f.slice/crio-a193a4fb9eb883e8ed29f339132060309ecaf340d211b5cbffd3dc000d7ee10a WatchSource:0}: Error finding container a193a4fb9eb883e8ed29f339132060309ecaf340d211b5cbffd3dc000d7ee10a: Status 404 returned error can't find the container with id a193a4fb9eb883e8ed29f339132060309ecaf340d211b5cbffd3dc000d7ee10a Jul 18 17:09:09.476865 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.476820 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:09.476970 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.476896 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:09.476970 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.476937 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:09.680320 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.680283 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerStarted","Data":"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d"} Jul 18 17:09:09.680539 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.680340 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerStarted","Data":"a193a4fb9eb883e8ed29f339132060309ecaf340d211b5cbffd3dc000d7ee10a"} Jul 18 17:09:09.699015 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:09.698967 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podStartSLOduration=0.698949317 podStartE2EDuration="698.949317ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:09:09.698545841 +0000 UTC m=+1954.837308506" watchObservedRunningTime="2026-07-18 17:09:09.698949317 +0000 UTC m=+1954.837711982" Jul 18 17:09:10.335876 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:10.335845 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:10.340586 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:10.340564 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:10.685492 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:10.685463 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:10.686341 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:10.686321 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:17.443576 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:17.443541 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:09:25.687246 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.687200 2567 patch_prober.go:28] interesting pod/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 17:09:25.687709 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.687285 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 17:09:25.761490 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.761461 2567 generic.go:356] "Generic (PLEG): container finished" podID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerID="3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d" exitCode=0 Jul 18 17:09:25.761651 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.761537 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerDied","Data":"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d"} Jul 18 17:09:25.761928 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.761915 2567 scope.go:117] "RemoveContainer" containerID="3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d" Jul 18 17:09:25.762943 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.762915 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:25.763016 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.762998 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:25.763052 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:25.763039 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:09:26.771727 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:26.771676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerStarted","Data":"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345"} Jul 18 17:09:27.336576 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:27.336453 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:27.336740 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:27.336711 2567 patch_prober.go:28] interesting pod/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 17:09:27.336791 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:27.336772 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 17:09:28.335885 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:28.335845 2567 patch_prober.go:28] interesting pod/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 17:09:28.336319 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:28.335925 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 17:09:29.335883 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:29.335844 2567 patch_prober.go:28] interesting pod/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 17:09:29.336294 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:29.335914 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 17:09:30.335895 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:30.335860 2567 patch_prober.go:28] interesting pod/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 17:09:30.336284 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:30.335913 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 17:09:31.339832 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:31.339802 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:31.340312 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:31.340189 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:31.340822 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:31.340804 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:50.243799 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:50.243765 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:09:53.032668 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:53.032631 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:53.033096 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:53.032858 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" containerID="cri-o://912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345" gracePeriod=30 Jul 18 17:09:58.279173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.279148 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:58.383233 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383199 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383233 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383239 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383262 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383283 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383325 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48p9c\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383341 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383408 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383475 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383432 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383520 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data\") pod \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\" (UID: \"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f\") " Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383541 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383657 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383790 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383805 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-certs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.383817 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383821 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-workload-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.384110 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.383864 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:09:58.384390 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.384367 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:09:58.385575 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.385551 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:09:58.385685 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.385604 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 17:09:58.385810 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.385787 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c" (OuterVolumeSpecName: "kube-api-access-48p9c") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "kube-api-access-48p9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:09:58.385937 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.385920 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" (UID: "d89ebe70-70e3-4c3a-9ec1-2f74e5de274f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:09:58.484345 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484308 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-data\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484345 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484338 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-podinfo\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484345 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484347 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-token\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484625 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484357 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48p9c\" (UniqueName: \"kubernetes.io/projected/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-kube-api-access-48p9c\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484625 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484368 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-credential-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484625 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484376 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istiod-ca-cert\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.484625 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.484384 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f-istio-envoy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:09:58.925943 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.925909 2567 generic.go:356] "Generic (PLEG): container finished" podID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerID="912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345" exitCode=0 Jul 18 17:09:58.925943 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.925946 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerDied","Data":"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345"} Jul 18 17:09:58.926189 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.925957 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" Jul 18 17:09:58.926189 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.925987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn" event={"ID":"d89ebe70-70e3-4c3a-9ec1-2f74e5de274f","Type":"ContainerDied","Data":"a193a4fb9eb883e8ed29f339132060309ecaf340d211b5cbffd3dc000d7ee10a"} Jul 18 17:09:58.926189 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.926008 2567 scope.go:117] "RemoveContainer" containerID="912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345" Jul 18 17:09:58.934925 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.934907 2567 scope.go:117] "RemoveContainer" containerID="3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d" Jul 18 17:09:58.942892 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.942876 2567 scope.go:117] "RemoveContainer" containerID="912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345" Jul 18 17:09:58.943149 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:09:58.943130 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345\": container with ID starting with 912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345 not found: ID does not exist" containerID="912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345" Jul 18 17:09:58.943194 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.943160 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345"} err="failed to get container status \"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345\": rpc error: code = NotFound desc = could not find container \"912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345\": container with ID starting with 912d3df4c9c7c62d01a7c8b0de83633631641366d9f1fca058ef600c68823345 not found: ID does not exist" Jul 18 17:09:58.943194 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.943177 2567 scope.go:117] "RemoveContainer" containerID="3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d" Jul 18 17:09:58.943408 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:09:58.943390 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d\": container with ID starting with 3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d not found: ID does not exist" containerID="3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d" Jul 18 17:09:58.943461 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.943417 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d"} err="failed to get container status \"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d\": rpc error: code = NotFound desc = could not find container \"3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d\": container with ID starting with 3578ffdefa5722c48fd6242e2b614e5a7ba048997366afbb07501280d154793d not found: ID does not exist" Jul 18 17:09:58.949039 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.949018 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:58.956243 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:58.956224 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b26f5ca7-openshift-default-6b6d485bc-dndtn"] Jul 18 17:09:59.280911 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:09:59.280834 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" path="/var/lib/kubelet/pods/d89ebe70-70e3-4c3a-9ec1-2f74e5de274f/volumes" Jul 18 17:10:01.245838 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:10:01.245795 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:10:34.743525 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:10:34.743482 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:11:35.400348 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:11:35.400269 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:11:35.404596 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:11:35.404570 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:12:00.947227 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:12:00.947184 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:12:12.848986 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:12:12.848950 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:12:24.943088 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:12:24.943054 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:12:42.739796 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:12:42.739760 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:13:17.141634 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:17.141593 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:13:26.183142 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:26.183109 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:13:38.071668 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:38.071630 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:13:41.175365 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175330 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6"] Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175688 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175701 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175722 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175727 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175735 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175740 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175796 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="54116eb9-b883-49de-8211-129a69c1d7f9" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175805 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.175863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.175811 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89ebe70-70e3-4c3a-9ec1-2f74e5de274f" containerName="istio-proxy" Jul 18 17:13:41.178721 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.178700 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:41.181351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.181329 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7rq8\"" Jul 18 17:13:41.181810 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.181795 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 17:13:41.197346 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.197312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6"] Jul 18 17:13:41.234941 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.234907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-259b7916-5mwz6\" (UID: \"63ac75d0-f43a-46f7-a620-85c9a103b2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:41.335775 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.335736 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-259b7916-5mwz6\" (UID: \"63ac75d0-f43a-46f7-a620-85c9a103b2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:41.336361 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.336342 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-259b7916-5mwz6\" (UID: \"63ac75d0-f43a-46f7-a620-85c9a103b2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:41.488982 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.488902 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:41.628257 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.628222 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6"] Jul 18 17:13:41.632095 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:13:41.632068 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ac75d0_f43a_46f7_a620_85c9a103b2c0.slice/crio-2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813 WatchSource:0}: Error finding container 2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813: Status 404 returned error can't find the container with id 2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813 Jul 18 17:13:41.633965 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.633949 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 17:13:41.966071 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:41.966034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" event={"ID":"63ac75d0-f43a-46f7-a620-85c9a103b2c0","Type":"ContainerStarted","Data":"2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813"} Jul 18 17:13:44.983030 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:44.982995 2567 generic.go:356] "Generic (PLEG): container finished" podID="63ac75d0-f43a-46f7-a620-85c9a103b2c0" containerID="de7b166abc3689deac420c9a2b877056796a436b0bc1d6d4b892dd40c57f403a" exitCode=0 Jul 18 17:13:44.983383 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:44.983040 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" event={"ID":"63ac75d0-f43a-46f7-a620-85c9a103b2c0","Type":"ContainerDied","Data":"de7b166abc3689deac420c9a2b877056796a436b0bc1d6d4b892dd40c57f403a"} Jul 18 17:13:46.114961 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.114932 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:46.183819 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.183784 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca\") pod \"63ac75d0-f43a-46f7-a620-85c9a103b2c0\" (UID: \"63ac75d0-f43a-46f7-a620-85c9a103b2c0\") " Jul 18 17:13:46.184178 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.184158 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63ac75d0-f43a-46f7-a620-85c9a103b2c0" (UID: "63ac75d0-f43a-46f7-a620-85c9a103b2c0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:13:46.284845 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.284767 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ac75d0-f43a-46f7-a620-85c9a103b2c0-maas-api-service-ca\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:13:46.993716 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.993679 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" event={"ID":"63ac75d0-f43a-46f7-a620-85c9a103b2c0","Type":"ContainerDied","Data":"2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813"} Jul 18 17:13:46.993716 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.993708 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6" Jul 18 17:13:46.993916 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:46.993730 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e088994745ab7eda70310ff80d83285d0cabad29ae0e820e631bc41dbf69813" Jul 18 17:13:55.184644 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:13:55.184609 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:14:01.304029 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:14:01.303935 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6"] Jul 18 17:14:01.314090 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:14:01.314059 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-259b7916-5mwz6"] Jul 18 17:14:03.281559 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:14:03.281522 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63ac75d0-f43a-46f7-a620-85c9a103b2c0" path="/var/lib/kubelet/pods/63ac75d0-f43a-46f7-a620-85c9a103b2c0/volumes" Jul 18 17:14:03.796884 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:14:03.796846 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:14:55.645819 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:14:55.645783 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:15:07.958130 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:07.958088 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:15:54.770749 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.770708 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:15:54.771205 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.771090 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ac75d0-f43a-46f7-a620-85c9a103b2c0" containerName="revoke-keys" Jul 18 17:15:54.771205 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.771101 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ac75d0-f43a-46f7-a620-85c9a103b2c0" containerName="revoke-keys" Jul 18 17:15:54.771205 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.771167 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ac75d0-f43a-46f7-a620-85c9a103b2c0" containerName="revoke-keys" Jul 18 17:15:54.774305 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.774283 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.776846 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.776826 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4554ed-openshift-default-dockercfg-whd2w\"" Jul 18 17:15:54.786439 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.786417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:15:54.831209 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831366 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831227 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831366 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831310 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831366 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831347 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831466 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831376 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831466 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831466 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831443 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvgpd\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831587 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.831587 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.831497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932257 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932424 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932424 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932496 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932465 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932576 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932542 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932631 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932585 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932631 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932611 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvgpd\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932806 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932690 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932806 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932726 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932806 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932994 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932876 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932994 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932918 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.932994 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.932935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.933586 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.933566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.934599 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.934569 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.935166 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.935149 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.940680 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.940654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:54.940775 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:54.940761 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvgpd\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd\") pod \"e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:55.087247 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.087161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:55.225058 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.225027 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:15:55.228345 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:15:55.228313 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42bedf2c_687d_4e51_b0e7_48f88b613137.slice/crio-d5ef0ccb8f7fca469b433784cd9b346435311390cff5ae43ccd6977c7f9e47c3 WatchSource:0}: Error finding container d5ef0ccb8f7fca469b433784cd9b346435311390cff5ae43ccd6977c7f9e47c3: Status 404 returned error can't find the container with id d5ef0ccb8f7fca469b433784cd9b346435311390cff5ae43ccd6977c7f9e47c3 Jul 18 17:15:55.230377 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.230347 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:15:55.230463 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.230412 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:15:55.230463 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.230442 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:15:55.343299 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.343208 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:15:55.592657 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.592621 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerStarted","Data":"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750"} Jul 18 17:15:55.592833 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.592676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerStarted","Data":"d5ef0ccb8f7fca469b433784cd9b346435311390cff5ae43ccd6977c7f9e47c3"} Jul 18 17:15:55.613193 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:55.613151 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podStartSLOduration=1.6131349240000001 podStartE2EDuration="1.613134924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:15:55.612095611 +0000 UTC m=+2360.750858311" watchObservedRunningTime="2026-07-18 17:15:55.613134924 +0000 UTC m=+2360.751897586" Jul 18 17:15:56.087758 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:56.087672 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:56.092273 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:56.092246 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:56.597873 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:56.597834 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:15:56.598858 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:15:56.598841 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:16:07.844499 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:07.844464 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:16:08.661714 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.661633 2567 generic.go:356] "Generic (PLEG): container finished" podID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerID="7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750" exitCode=0 Jul 18 17:16:08.661714 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.661693 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerDied","Data":"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750"} Jul 18 17:16:08.662061 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.662047 2567 scope.go:117] "RemoveContainer" containerID="7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750" Jul 18 17:16:08.663080 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.663050 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:16:08.663151 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.663133 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:16:08.663200 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:08.663187 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:16:09.668574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:09.668534 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerStarted","Data":"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37"} Jul 18 17:16:10.087895 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:10.087803 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:16:10.088884 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:10.088861 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 18 17:16:10.088934 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:10.088914 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 18 17:16:11.088284 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:11.088250 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 18 17:16:11.088681 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:11.088306 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 18 17:16:11.598687 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:11.598659 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:16:12.087877 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:12.087839 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 18 17:16:12.088061 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:12.087895 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 18 17:16:13.088290 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:13.088258 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 18 17:16:13.088689 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:13.088309 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 18 17:16:14.092100 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:14.092070 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:16:14.092914 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:14.092893 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:16:18.853768 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:18.853733 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:16:35.433140 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:35.433107 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:16:35.438223 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:35.438204 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:16:43.744618 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:43.744585 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:16:57.244239 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:16:57.244154 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:17:12.125846 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.125815 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw"] Jul 18 17:17:12.129110 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.129091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:12.131463 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.131440 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 17:17:12.131610 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.131442 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7rq8\"" Jul 18 17:17:12.135598 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.135575 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw"] Jul 18 17:17:12.171544 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.171494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-189241-pmgzw\" (UID: \"0a1aefd7-1340-42e6-8750-60d23e9fb795\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:12.272860 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.272824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-189241-pmgzw\" (UID: \"0a1aefd7-1340-42e6-8750-60d23e9fb795\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:12.273467 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.273446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-189241-pmgzw\" (UID: \"0a1aefd7-1340-42e6-8750-60d23e9fb795\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:12.441844 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.441769 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:12.572621 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.572595 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw"] Jul 18 17:17:12.575032 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:17:12.575008 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1aefd7_1340_42e6_8750_60d23e9fb795.slice/crio-3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad WatchSource:0}: Error finding container 3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad: Status 404 returned error can't find the container with id 3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad Jul 18 17:17:12.964238 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.964206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" event={"ID":"0a1aefd7-1340-42e6-8750-60d23e9fb795","Type":"ContainerStarted","Data":"00aa72215a8da118e36f51c1e0273761fd1dcd46c512c04e4a95fd169bcf2d44"} Jul 18 17:17:12.964379 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.964256 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" event={"ID":"0a1aefd7-1340-42e6-8750-60d23e9fb795","Type":"ContainerStarted","Data":"3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad"} Jul 18 17:17:12.981435 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:12.981386 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" podStartSLOduration=0.981370751 podStartE2EDuration="981.370751ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:17:12.97919411 +0000 UTC m=+2438.117956792" watchObservedRunningTime="2026-07-18 17:17:12.981370751 +0000 UTC m=+2438.120133391" Jul 18 17:17:13.970449 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:13.970416 2567 generic.go:356] "Generic (PLEG): container finished" podID="0a1aefd7-1340-42e6-8750-60d23e9fb795" containerID="00aa72215a8da118e36f51c1e0273761fd1dcd46c512c04e4a95fd169bcf2d44" exitCode=0 Jul 18 17:17:13.970863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:13.970496 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" event={"ID":"0a1aefd7-1340-42e6-8750-60d23e9fb795","Type":"ContainerDied","Data":"00aa72215a8da118e36f51c1e0273761fd1dcd46c512c04e4a95fd169bcf2d44"} Jul 18 17:17:15.109398 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.109371 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:15.199266 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.199236 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca\") pod \"0a1aefd7-1340-42e6-8750-60d23e9fb795\" (UID: \"0a1aefd7-1340-42e6-8750-60d23e9fb795\") " Jul 18 17:17:15.199701 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.199677 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a1aefd7-1340-42e6-8750-60d23e9fb795" (UID: "0a1aefd7-1340-42e6-8750-60d23e9fb795"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:17:15.300870 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.300845 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aefd7-1340-42e6-8750-60d23e9fb795-maas-api-service-ca\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:17:15.980519 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.980478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" event={"ID":"0a1aefd7-1340-42e6-8750-60d23e9fb795","Type":"ContainerDied","Data":"3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad"} Jul 18 17:17:15.980699 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.980562 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f0683f0945cf96c66e19dcbf8e7bf6fabcf81508ac73e932cde664abc5813ad" Jul 18 17:17:15.980699 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:15.980528 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw" Jul 18 17:17:16.246052 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:16.245968 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:17:27.219198 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:27.219160 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw"] Jul 18 17:17:27.221051 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:27.221028 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-189241-pmgzw"] Jul 18 17:17:27.280703 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:27.280669 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a1aefd7-1340-42e6-8750-60d23e9fb795" path="/var/lib/kubelet/pods/0a1aefd7-1340-42e6-8750-60d23e9fb795/volumes" Jul 18 17:17:31.050852 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:31.050819 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:17:47.243769 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:47.243735 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:17:55.443428 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:55.443393 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:17:55.443908 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:55.443664 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" containerID="cri-o://845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37" gracePeriod=30 Jul 18 17:17:59.095329 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:59.095288 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 17:17:59.095720 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:17:59.095371 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 17:18:00.699193 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.699165 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:18:00.823412 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823327 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823412 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823406 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823438 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823493 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823528 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvgpd\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823553 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823579 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823606 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.823677 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823630 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert\") pod \"42bedf2c-687d-4e51-b0e7-48f88b613137\" (UID: \"42bedf2c-687d-4e51-b0e7-48f88b613137\") " Jul 18 17:18:00.824079 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823752 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:18:00.824079 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.823922 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-credential-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.824182 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.824119 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:18:00.824182 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.824164 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:18:00.824398 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.824374 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:18:00.824685 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.824664 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data" (OuterVolumeSpecName: "istio-data") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:18:00.825698 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.825674 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token" (OuterVolumeSpecName: "istio-token") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:18:00.825863 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.825843 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 17:18:00.826085 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.826068 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:18:00.826161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.826147 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd" (OuterVolumeSpecName: "kube-api-access-qvgpd") pod "42bedf2c-687d-4e51-b0e7-48f88b613137" (UID: "42bedf2c-687d-4e51-b0e7-48f88b613137"). InnerVolumeSpecName "kube-api-access-qvgpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:18:00.924799 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924764 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-token\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.924799 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924792 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-data\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.924799 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924802 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvgpd\" (UniqueName: \"kubernetes.io/projected/42bedf2c-687d-4e51-b0e7-48f88b613137-kube-api-access-qvgpd\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.925032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924813 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-envoy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.925032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924822 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-certs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.925032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924829 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42bedf2c-687d-4e51-b0e7-48f88b613137-workload-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.925032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924837 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42bedf2c-687d-4e51-b0e7-48f88b613137-istiod-ca-cert\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:00.925032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:00.924844 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42bedf2c-687d-4e51-b0e7-48f88b613137-istio-podinfo\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:01.198986 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.198947 2567 generic.go:356] "Generic (PLEG): container finished" podID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerID="845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37" exitCode=0 Jul 18 17:18:01.198986 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.198979 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerDied","Data":"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37"} Jul 18 17:18:01.199202 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.199034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" event={"ID":"42bedf2c-687d-4e51-b0e7-48f88b613137","Type":"ContainerDied","Data":"d5ef0ccb8f7fca469b433784cd9b346435311390cff5ae43ccd6977c7f9e47c3"} Jul 18 17:18:01.199202 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.199060 2567 scope.go:117] "RemoveContainer" containerID="845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37" Jul 18 17:18:01.199202 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.198994 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27" Jul 18 17:18:01.208749 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.208727 2567 scope.go:117] "RemoveContainer" containerID="7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750" Jul 18 17:18:01.216836 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.216805 2567 scope.go:117] "RemoveContainer" containerID="845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37" Jul 18 17:18:01.217138 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:18:01.217118 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37\": container with ID starting with 845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37 not found: ID does not exist" containerID="845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37" Jul 18 17:18:01.217185 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.217149 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37"} err="failed to get container status \"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37\": rpc error: code = NotFound desc = could not find container \"845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37\": container with ID starting with 845d39562a13a51db78adfcca6725ba325987178a09a1b424f17237739a11a37 not found: ID does not exist" Jul 18 17:18:01.217185 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.217171 2567 scope.go:117] "RemoveContainer" containerID="7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750" Jul 18 17:18:01.217435 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:18:01.217415 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750\": container with ID starting with 7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750 not found: ID does not exist" containerID="7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750" Jul 18 17:18:01.217557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.217440 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750"} err="failed to get container status \"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750\": rpc error: code = NotFound desc = could not find container \"7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750\": container with ID starting with 7e5ef4b758b6ab112b585a5aa2b6e4dbf9419ef3963cb96b374ca0d69591a750 not found: ID does not exist" Jul 18 17:18:01.221426 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.221403 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:18:01.228549 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.228503 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4554ed-openshift-default-678f6f7569-w4z27"] Jul 18 17:18:01.280724 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:01.280685 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" path="/var/lib/kubelet/pods/42bedf2c-687d-4e51-b0e7-48f88b613137/volumes" Jul 18 17:18:04.544960 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.544916 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:18:04.545502 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545485 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:04.545502 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545521 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:04.545642 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545572 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a1aefd7-1340-42e6-8750-60d23e9fb795" containerName="revoke-keys" Jul 18 17:18:04.545642 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545582 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a1aefd7-1340-42e6-8750-60d23e9fb795" containerName="revoke-keys" Jul 18 17:18:04.545745 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545672 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a1aefd7-1340-42e6-8750-60d23e9fb795" containerName="revoke-keys" Jul 18 17:18:04.545745 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545687 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:04.545745 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.545697 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:04.550535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.550497 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.553248 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.553220 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-376c15-openshift-default-dockercfg-2lvhv\"" Jul 18 17:18:04.559983 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.559962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:18:04.657462 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657687 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657485 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657687 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657576 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657687 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657687 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657633 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcshv\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657899 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657899 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657804 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657899 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.657899 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.657847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.758736 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758700 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.758937 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.758937 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.758937 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.758937 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759144 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759144 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.758989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759144 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759144 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcshv\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759174 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759470 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759447 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759470 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759459 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.759770 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.759751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.761143 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.761124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.761395 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.761373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.766398 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.766378 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.766790 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.766769 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcshv\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv\") pod \"e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.862421 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.862338 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:04.947903 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.947873 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:18:04.996159 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:04.996134 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:18:05.001450 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:18:05.001423 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abb3155_119d_458b_be8c_92a37bf9fb1b.slice/crio-0acbbcfb409d673121934d155ab15ecef25f1642ca7a8120990478b01e631d8f WatchSource:0}: Error finding container 0acbbcfb409d673121934d155ab15ecef25f1642ca7a8120990478b01e631d8f: Status 404 returned error can't find the container with id 0acbbcfb409d673121934d155ab15ecef25f1642ca7a8120990478b01e631d8f Jul 18 17:18:05.003661 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.003628 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:05.003765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.003705 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:05.003765 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.003750 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:05.229958 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.229924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerStarted","Data":"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92"} Jul 18 17:18:05.230119 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.229980 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerStarted","Data":"0acbbcfb409d673121934d155ab15ecef25f1642ca7a8120990478b01e631d8f"} Jul 18 17:18:05.249270 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.249223 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podStartSLOduration=1.249206717 podStartE2EDuration="1.249206717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:18:05.248143436 +0000 UTC m=+2490.386906114" watchObservedRunningTime="2026-07-18 17:18:05.249206717 +0000 UTC m=+2490.387969379" Jul 18 17:18:05.862618 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:05.862584 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:06.867397 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:06.867368 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:07.240950 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:07.240855 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:07.241991 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:07.241960 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:17.854077 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:17.854035 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:18:19.299660 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.299625 2567 generic.go:356] "Generic (PLEG): container finished" podID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerID="70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92" exitCode=0 Jul 18 17:18:19.300068 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.299699 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerDied","Data":"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92"} Jul 18 17:18:19.300111 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.300090 2567 scope.go:117] "RemoveContainer" containerID="70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92" Jul 18 17:18:19.301175 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.301147 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:19.301265 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.301213 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:19.301265 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.301245 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:18:19.562205 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.562114 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv"] Jul 18 17:18:19.562595 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.562578 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:19.562595 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.562596 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bedf2c-687d-4e51-b0e7-48f88b613137" containerName="istio-proxy" Jul 18 17:18:19.565686 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.565666 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:19.567983 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.567960 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7rq8\"" Jul 18 17:18:19.568098 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.567970 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 17:18:19.572293 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.572269 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv"] Jul 18 17:18:19.602132 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.602097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-73d394-56hvv\" (UID: \"f4c61f51-cdf1-46a6-9fe8-e32242ddf123\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:19.703700 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.703664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-73d394-56hvv\" (UID: \"f4c61f51-cdf1-46a6-9fe8-e32242ddf123\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:19.704263 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.704245 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-73d394-56hvv\" (UID: \"f4c61f51-cdf1-46a6-9fe8-e32242ddf123\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:19.878637 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:19.878594 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:20.285060 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.285033 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv"] Jul 18 17:18:20.286656 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:18:20.286634 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c61f51_cdf1_46a6_9fe8_e32242ddf123.slice/crio-bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f WatchSource:0}: Error finding container bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f: Status 404 returned error can't find the container with id bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f Jul 18 17:18:20.316905 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.315948 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerStarted","Data":"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef"} Jul 18 17:18:20.318772 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.318695 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" event={"ID":"f4c61f51-cdf1-46a6-9fe8-e32242ddf123","Type":"ContainerStarted","Data":"bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f"} Jul 18 17:18:20.863366 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.863323 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:20.864503 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.864478 2567 patch_prober.go:28] interesting pod/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 18 17:18:20.864631 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:20.864558 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 18 17:18:21.329902 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:21.329863 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" event={"ID":"f4c61f51-cdf1-46a6-9fe8-e32242ddf123","Type":"ContainerStarted","Data":"6644ec7dfc9f0f496a0783f691a28e6c3d9d5a8595eeb032d111d58d5ed7cfb2"} Jul 18 17:18:21.345625 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:21.345576 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" podStartSLOduration=2.345561073 podStartE2EDuration="2.345561073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:18:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:18:21.344606165 +0000 UTC m=+2506.483368845" watchObservedRunningTime="2026-07-18 17:18:21.345561073 +0000 UTC m=+2506.484323736" Jul 18 17:18:21.862926 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:21.862886 2567 patch_prober.go:28] interesting pod/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 18 17:18:21.863143 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:21.862949 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 18 17:18:22.241664 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:22.241576 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:22.335247 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:22.335210 2567 generic.go:356] "Generic (PLEG): container finished" podID="f4c61f51-cdf1-46a6-9fe8-e32242ddf123" containerID="6644ec7dfc9f0f496a0783f691a28e6c3d9d5a8595eeb032d111d58d5ed7cfb2" exitCode=0 Jul 18 17:18:22.335748 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:22.335300 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" event={"ID":"f4c61f51-cdf1-46a6-9fe8-e32242ddf123","Type":"ContainerDied","Data":"6644ec7dfc9f0f496a0783f691a28e6c3d9d5a8595eeb032d111d58d5ed7cfb2"} Jul 18 17:18:22.863105 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:22.863068 2567 patch_prober.go:28] interesting pod/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 18 17:18:22.863287 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:22.863123 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 18 17:18:23.487421 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.487398 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:23.556920 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.556884 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca\") pod \"f4c61f51-cdf1-46a6-9fe8-e32242ddf123\" (UID: \"f4c61f51-cdf1-46a6-9fe8-e32242ddf123\") " Jul 18 17:18:23.557223 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.557201 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4c61f51-cdf1-46a6-9fe8-e32242ddf123" (UID: "f4c61f51-cdf1-46a6-9fe8-e32242ddf123"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:18:23.657768 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.657734 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4c61f51-cdf1-46a6-9fe8-e32242ddf123-maas-api-service-ca\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:18:23.863706 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.863666 2567 patch_prober.go:28] interesting pod/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 18 17:18:23.863879 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:23.863722 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 18 17:18:24.346171 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:24.346145 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" Jul 18 17:18:24.346356 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:24.346178 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv" event={"ID":"f4c61f51-cdf1-46a6-9fe8-e32242ddf123","Type":"ContainerDied","Data":"bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f"} Jul 18 17:18:24.346356 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:24.346223 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc3771f831a609ae143f691a830febc531a1ffcb2475300450d6e807964a3f9f" Jul 18 17:18:24.867425 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:24.867391 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:24.868312 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:24.868293 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:18:37.543887 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:37.543853 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:18:39.637237 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:39.637196 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv"] Jul 18 17:18:39.638975 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:39.638954 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-73d394-56hvv"] Jul 18 17:18:41.280346 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:18:41.280311 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4c61f51-cdf1-46a6-9fe8-e32242ddf123" path="/var/lib/kubelet/pods/f4c61f51-cdf1-46a6-9fe8-e32242ddf123/volumes" Jul 18 17:19:06.740894 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:06.740858 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:19:12.779484 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:12.779448 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:19:12.779899 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:12.779713 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" containerID="cri-o://6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef" gracePeriod=30 Jul 18 17:19:18.029816 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.029793 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:19:18.149380 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149343 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149399 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149454 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149486 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149534 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcshv\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149559 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149588 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149933 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149627 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149933 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149649 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data\") pod \"3abb3155-119d-458b-be8c-92a37bf9fb1b\" (UID: \"3abb3155-119d-458b-be8c-92a37bf9fb1b\") " Jul 18 17:19:18.149933 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149682 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:19:18.149933 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149813 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:19:18.149933 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149903 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:19:18.150182 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.149980 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-certs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.150182 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.150000 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-workload-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.150182 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.150015 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-credential-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.150332 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.150259 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:19:18.150659 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.150632 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:19:18.151779 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.151750 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:19:18.151779 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.151760 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv" (OuterVolumeSpecName: "kube-api-access-kcshv") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "kube-api-access-kcshv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:19:18.151779 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.151769 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:19:18.151924 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.151795 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3abb3155-119d-458b-be8c-92a37bf9fb1b" (UID: "3abb3155-119d-458b-be8c-92a37bf9fb1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 17:19:18.250547 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250499 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-envoy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.250547 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250546 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-podinfo\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.250704 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250556 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcshv\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-kube-api-access-kcshv\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.250704 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250567 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abb3155-119d-458b-be8c-92a37bf9fb1b-istiod-ca-cert\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.250704 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250575 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-token\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.250704 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.250582 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abb3155-119d-458b-be8c-92a37bf9fb1b-istio-data\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:19:18.608166 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.608078 2567 generic.go:356] "Generic (PLEG): container finished" podID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerID="6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef" exitCode=0 Jul 18 17:19:18.608166 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.608123 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerDied","Data":"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef"} Jul 18 17:19:18.608166 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.608128 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" Jul 18 17:19:18.608409 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.608178 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd" event={"ID":"3abb3155-119d-458b-be8c-92a37bf9fb1b","Type":"ContainerDied","Data":"0acbbcfb409d673121934d155ab15ecef25f1642ca7a8120990478b01e631d8f"} Jul 18 17:19:18.608409 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.608208 2567 scope.go:117] "RemoveContainer" containerID="6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef" Jul 18 17:19:18.623665 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.623643 2567 scope.go:117] "RemoveContainer" containerID="70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92" Jul 18 17:19:18.631870 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.631854 2567 scope.go:117] "RemoveContainer" containerID="6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef" Jul 18 17:19:18.632180 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:19:18.632159 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef\": container with ID starting with 6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef not found: ID does not exist" containerID="6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef" Jul 18 17:19:18.632238 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.632187 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef"} err="failed to get container status \"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef\": rpc error: code = NotFound desc = could not find container \"6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef\": container with ID starting with 6d50c701d416a02048372d2926d1c04d28f0bd5e2681a9a37d6a8eecb8d961ef not found: ID does not exist" Jul 18 17:19:18.632238 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.632205 2567 scope.go:117] "RemoveContainer" containerID="70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92" Jul 18 17:19:18.632454 ip-10-0-139-1 kubenswrapper[2567]: E0718 17:19:18.632435 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92\": container with ID starting with 70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92 not found: ID does not exist" containerID="70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92" Jul 18 17:19:18.632529 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.632465 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92"} err="failed to get container status \"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92\": rpc error: code = NotFound desc = could not find container \"70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92\": container with ID starting with 70d566597aa975cdbeed6071b705db459efc28c9aafde75871419549b0a15c92 not found: ID does not exist" Jul 18 17:19:18.632888 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.632870 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:19:18.637083 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:18.637058 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-376c15-openshift-default-697d69b8bc-fgmwd"] Jul 18 17:19:19.280371 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:19.280337 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" path="/var/lib/kubelet/pods/3abb3155-119d-458b-be8c-92a37bf9fb1b/volumes" Jul 18 17:19:21.832916 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.832877 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:19:21.833440 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833424 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:19:21.833535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833443 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:19:21.833535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833458 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c61f51-cdf1-46a6-9fe8-e32242ddf123" containerName="revoke-keys" Jul 18 17:19:21.833535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833467 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c61f51-cdf1-46a6-9fe8-e32242ddf123" containerName="revoke-keys" Jul 18 17:19:21.833535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833481 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:19:21.833535 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833490 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:19:21.833693 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833588 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c61f51-cdf1-46a6-9fe8-e32242ddf123" containerName="revoke-keys" Jul 18 17:19:21.833693 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.833602 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:19:21.838570 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.838546 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.840943 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.840915 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-497f21-openshift-default-dockercfg-65smb\"" Jul 18 17:19:21.850958 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.850939 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:19:21.983979 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.983947 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.983996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmz2z\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984100 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984161 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984123 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984457 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984457 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984237 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984457 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984265 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:21.984457 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:21.984293 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.084820 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.084820 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmz2z\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.084820 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.084820 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084821 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084885 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084931 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.084982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085037 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085679 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085616 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085679 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085628 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085802 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085781 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.085883 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.085862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.087340 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.087314 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.087936 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.087917 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.093607 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.093585 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.093735 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.093714 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmz2z\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z\") pod \"e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.148920 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.148881 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:22.292537 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.292489 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:19:22.294703 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:19:22.294676 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47eb8b55_0ee2_48a4_810e_4b8776853719.slice/crio-46e24f03bc07441bbad09ee46579f3fa388e29c35984f6baa786a9bf02471b47 WatchSource:0}: Error finding container 46e24f03bc07441bbad09ee46579f3fa388e29c35984f6baa786a9bf02471b47: Status 404 returned error can't find the container with id 46e24f03bc07441bbad09ee46579f3fa388e29c35984f6baa786a9bf02471b47 Jul 18 17:19:22.296818 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.296790 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:22.296892 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.296852 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:22.296892 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.296879 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:22.630583 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.630543 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerStarted","Data":"8a7be0a73f0de24678323546fd0806ce2bc523217a2d544706cb5deba5cbc44b"} Jul 18 17:19:22.630761 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.630608 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerStarted","Data":"46e24f03bc07441bbad09ee46579f3fa388e29c35984f6baa786a9bf02471b47"} Jul 18 17:19:22.672010 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:22.671953 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podStartSLOduration=1.6719354910000002 podStartE2EDuration="1.671935491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 17:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:19:22.669794512 +0000 UTC m=+2567.808557195" watchObservedRunningTime="2026-07-18 17:19:22.671935491 +0000 UTC m=+2567.810698153" Jul 18 17:19:23.149568 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:23.149536 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:23.154305 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:23.154281 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:23.635632 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:23.635601 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:23.636597 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:23.636575 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:24.949832 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:24.949793 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:19:35.739843 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:35.739759 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:19:38.761645 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:38.760542 2567 patch_prober.go:28] interesting pod/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:43628->10.134.0.38:15021: read: connection reset by peer" start-of-body= Jul 18 17:19:38.761645 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:38.760608 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:43628->10.134.0.38:15021: read: connection reset by peer" Jul 18 17:19:39.715225 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.715193 2567 generic.go:356] "Generic (PLEG): container finished" podID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerID="8a7be0a73f0de24678323546fd0806ce2bc523217a2d544706cb5deba5cbc44b" exitCode=0 Jul 18 17:19:39.715404 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.715247 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerDied","Data":"8a7be0a73f0de24678323546fd0806ce2bc523217a2d544706cb5deba5cbc44b"} Jul 18 17:19:39.715633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.715620 2567 scope.go:117] "RemoveContainer" containerID="8a7be0a73f0de24678323546fd0806ce2bc523217a2d544706cb5deba5cbc44b" Jul 18 17:19:39.716646 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.716614 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:39.716732 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.716718 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:39.716773 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:39.716765 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 17:19:40.721985 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:40.721948 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerStarted","Data":"8592fd6ba1edba93067233a0e294c1bf438e9cd4d8df16ca6650d3e49a5c7641"} Jul 18 17:19:41.150085 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:41.150045 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:41.150281 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:41.150256 2567 patch_prober.go:28] interesting pod/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 18 17:19:41.150328 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:41.150307 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 18 17:19:42.150126 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:42.150092 2567 patch_prober.go:28] interesting pod/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 18 17:19:42.150556 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:42.150149 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 18 17:19:43.149655 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:43.149612 2567 patch_prober.go:28] interesting pod/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 18 17:19:43.149838 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:43.149677 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 18 17:19:44.150236 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:44.150198 2567 patch_prober.go:28] interesting pod/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 18 17:19:44.150650 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:44.150262 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 18 17:19:45.153105 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:45.153066 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:45.153536 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:45.153368 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:45.154387 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:45.154367 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:19:46.738835 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:19:46.738806 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:20:12.937593 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:20:12.937492 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:20:35.487943 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:20:35.487913 2567 scope.go:117] "RemoveContainer" containerID="de7b166abc3689deac420c9a2b877056796a436b0bc1d6d4b892dd40c57f403a" Jul 18 17:21:01.561245 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:01.561208 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:21:20.543613 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:20.543576 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:21:35.468039 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:35.467966 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:21:35.474287 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:35.474267 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:21:38.743815 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:38.743772 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:21:53.644850 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:21:53.644815 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:22:11.640584 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:22:11.640541 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:22:30.346770 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:22:30.346736 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:22:47.549877 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:22:47.549846 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:23:07.146619 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:23:07.146539 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:23:35.521287 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:23:35.521256 2567 scope.go:117] "RemoveContainer" containerID="00aa72215a8da118e36f51c1e0273761fd1dcd46c512c04e4a95fd169bcf2d44" Jul 18 17:23:51.809337 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:23:51.809289 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:24:07.057026 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:24:07.056991 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:24:25.617480 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:24:25.617439 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:24:35.539966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:24:35.539884 2567 scope.go:117] "RemoveContainer" containerID="6644ec7dfc9f0f496a0783f691a28e6c3d9d5a8595eeb032d111d58d5ed7cfb2" Jul 18 17:24:37.546201 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:24:37.546164 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:24:53.641265 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:24:53.641228 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:25:14.542093 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:25:14.542053 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:25:29.143088 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:25:29.143057 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:25:43.646608 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:25:43.646576 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:25:58.644565 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:25:58.644469 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:26:10.906454 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.906419 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n"] Jul 18 17:26:10.906917 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.906902 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abb3155-119d-458b-be8c-92a37bf9fb1b" containerName="istio-proxy" Jul 18 17:26:10.909976 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.909961 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:10.912591 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.912568 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7rq8\"" Jul 18 17:26:10.912725 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.912592 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 17:26:10.915814 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.915793 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n"] Jul 18 17:26:10.998597 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:10.998558 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n\" (UID: \"328c5450-456b-42b2-b886-d6dabc6d7d2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:11.099755 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:11.099718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n\" (UID: \"328c5450-456b-42b2-b886-d6dabc6d7d2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:11.100354 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:11.100333 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n\" (UID: \"328c5450-456b-42b2-b886-d6dabc6d7d2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:11.221760 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:11.221665 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:11.355792 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:11.355679 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n"] Jul 18 17:26:11.358227 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:26:11.358192 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328c5450_456b_42b2_b886_d6dabc6d7d2c.slice/crio-334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc WatchSource:0}: Error finding container 334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc: Status 404 returned error can't find the container with id 334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc Jul 18 17:26:11.598999 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:11.598960 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" event={"ID":"328c5450-456b-42b2-b886-d6dabc6d7d2c","Type":"ContainerStarted","Data":"334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc"} Jul 18 17:26:12.604610 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:12.604570 2567 generic.go:356] "Generic (PLEG): container finished" podID="328c5450-456b-42b2-b886-d6dabc6d7d2c" containerID="f1574b6e0159d7cc4e15704ccec939995314f221d04bdd1a84bbcec30a463c04" exitCode=0 Jul 18 17:26:12.604994 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:12.604622 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" event={"ID":"328c5450-456b-42b2-b886-d6dabc6d7d2c","Type":"ContainerDied","Data":"f1574b6e0159d7cc4e15704ccec939995314f221d04bdd1a84bbcec30a463c04"} Jul 18 17:26:13.739313 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:13.739289 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:13.827675 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:13.827637 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca\") pod \"328c5450-456b-42b2-b886-d6dabc6d7d2c\" (UID: \"328c5450-456b-42b2-b886-d6dabc6d7d2c\") " Jul 18 17:26:13.827994 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:13.827972 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "328c5450-456b-42b2-b886-d6dabc6d7d2c" (UID: "328c5450-456b-42b2-b886-d6dabc6d7d2c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:26:13.928984 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:13.928940 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/328c5450-456b-42b2-b886-d6dabc6d7d2c-maas-api-service-ca\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:26:14.615252 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:14.615212 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" event={"ID":"328c5450-456b-42b2-b886-d6dabc6d7d2c","Type":"ContainerDied","Data":"334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc"} Jul 18 17:26:14.615252 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:14.615240 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n" Jul 18 17:26:14.615452 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:14.615272 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="334d5237480ce9f1f6780774c014954a49121732fcf35dbb6075bce0661b0ffc" Jul 18 17:26:24.336738 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:24.336702 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:26:25.988212 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:25.988176 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n"] Jul 18 17:26:25.991066 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:25.991040 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-78538d-c9z8n"] Jul 18 17:26:27.281145 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:27.281109 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328c5450-456b-42b2-b886-d6dabc6d7d2c" path="/var/lib/kubelet/pods/328c5450-456b-42b2-b886-d6dabc6d7d2c/volumes" Jul 18 17:26:35.497267 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:35.497236 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:26:35.504019 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:35.503998 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-smlc5_72d5a44e-acb8-4ab5-99d3-ddd9c36704df/ovn-acl-logging/0.log" Jul 18 17:26:46.341478 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:46.341442 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jvx76"] Jul 18 17:26:54.631752 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:54.631721 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:26:54.632162 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:54.631934 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" containerID="cri-o://8592fd6ba1edba93067233a0e294c1bf438e9cd4d8df16ca6650d3e49a5c7641" gracePeriod=30 Jul 18 17:26:59.836638 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.836606 2567 generic.go:356] "Generic (PLEG): container finished" podID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerID="8592fd6ba1edba93067233a0e294c1bf438e9cd4d8df16ca6650d3e49a5c7641" exitCode=0 Jul 18 17:26:59.837078 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.836677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerDied","Data":"8592fd6ba1edba93067233a0e294c1bf438e9cd4d8df16ca6650d3e49a5c7641"} Jul 18 17:26:59.837078 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.836740 2567 scope.go:117] "RemoveContainer" containerID="8a7be0a73f0de24678323546fd0806ce2bc523217a2d544706cb5deba5cbc44b" Jul 18 17:26:59.884449 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.884424 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:26:59.940931 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.940838 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.940931 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.940899 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.940954 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmz2z\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.940971 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.940987 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941003 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941030 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941056 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941164 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941157 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert\") pod \"47eb8b55-0ee2-48a4-810e-4b8776853719\" (UID: \"47eb8b55-0ee2-48a4-810e-4b8776853719\") " Jul 18 17:26:59.941574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941224 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:26:59.941574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941381 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:26:59.941574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941428 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:26:59.941574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941502 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-credential-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:26:59.941574 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941543 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-certs\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:26:59.941847 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941711 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 17:26:59.941998 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.941974 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data" (OuterVolumeSpecName: "istio-data") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:26:59.943290 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.943271 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 17:26:59.943384 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.943351 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token" (OuterVolumeSpecName: "istio-token") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:26:59.943534 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.943498 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z" (OuterVolumeSpecName: "kube-api-access-pmz2z") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "kube-api-access-pmz2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 17:26:59.943595 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:26:59.943531 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "47eb8b55-0ee2-48a4-810e-4b8776853719" (UID: "47eb8b55-0ee2-48a4-810e-4b8776853719"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 17:27:00.042351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042312 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-podinfo\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042345 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmz2z\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-kube-api-access-pmz2z\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042351 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042361 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-data\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042372 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-workload-socket\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042383 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-token\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042393 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47eb8b55-0ee2-48a4-810e-4b8776853719-istio-envoy\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.042633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.042403 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47eb8b55-0ee2-48a4-810e-4b8776853719-istiod-ca-cert\") on node \"ip-10-0-139-1.ec2.internal\" DevicePath \"\"" Jul 18 17:27:00.584385 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.584356 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ghnr4_aeb58dd5-641b-4d4b-9da9-52f4f543d061/manager/0.log" Jul 18 17:27:00.703679 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.703649 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rjw9z_cdd95720-ff0e-4bbb-86e3-18e356908c76/manager/0.log" Jul 18 17:27:00.843095 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.843006 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" Jul 18 17:27:00.843095 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.843036 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds" event={"ID":"47eb8b55-0ee2-48a4-810e-4b8776853719","Type":"ContainerDied","Data":"46e24f03bc07441bbad09ee46579f3fa388e29c35984f6baa786a9bf02471b47"} Jul 18 17:27:00.843597 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.843097 2567 scope.go:117] "RemoveContainer" containerID="8592fd6ba1edba93067233a0e294c1bf438e9cd4d8df16ca6650d3e49a5c7641" Jul 18 17:27:00.865032 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.865001 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:27:00.874314 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.869830 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-497f21-openshift-default-584b476c79-vz4ds"] Jul 18 17:27:00.937775 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:00.937746 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-697w9_805b14f7-698f-43a7-8ef3-31b15d16dc3d/server/0.log" Jul 18 17:27:01.280097 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:01.280066 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" path="/var/lib/kubelet/pods/47eb8b55-0ee2-48a4-810e-4b8776853719/volumes" Jul 18 17:27:01.428837 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:01.428812 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c768ddb5d-wp5gw_08e06ba3-2865-4762-a85e-c2f0906bed81/copy-manifests/0.log" Jul 18 17:27:01.435948 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:01.435919 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c768ddb5d-wp5gw_08e06ba3-2865-4762-a85e-c2f0906bed81/manager/0.log" Jul 18 17:27:03.660158 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:03.660126 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4rl97_010700c4-200a-4c7b-abe9-247f22afeadd/manager/0.log" Jul 18 17:27:04.133128 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:04.133094 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jvx76_77c17dfc-d6e9-4680-892b-e0fb9075bd89/limitador/0.log" Jul 18 17:27:04.599238 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:04.599161 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rpn86_50c0faeb-95d1-489b-b792-e67b37b5d0c6/istio-proxy/0.log" Jul 18 17:27:04.715550 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:04.715503 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6whd6_8d1e349e-3ecd-46d1-beda-6c7a76d82894/discovery/0.log" Jul 18 17:27:04.827724 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:04.827700 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c475ff8c-bv4fz_932ff878-af0b-4930-b433-b066253f1e87/kube-auth-proxy/0.log" Jul 18 17:27:05.057132 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:05.057105 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/1.log" Jul 18 17:27:05.872749 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:05.872725 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qb98x_0f3828c8-4171-45cc-85a4-b2be0a46411a/main/0.log" Jul 18 17:27:06.206613 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:06.206503 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rscjm_d5ad940d-06f2-4794-b277-df8728231542/main/0.log" Jul 18 17:27:14.716325 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:14.716292 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sc698_c8a69b23-4563-44fd-825b-d3a148575aec/global-pull-secret-syncer/0.log" Jul 18 17:27:14.923763 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:14.923732 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wq4kr_278663ef-6a36-4fde-9a07-bd028a1e19c3/konnectivity-agent/0.log" Jul 18 17:27:14.966977 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:14.966887 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-1.ec2.internal_4359bec0e352431350db386bdadf6398/haproxy/0.log" Jul 18 17:27:20.711333 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:20.711296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4rl97_010700c4-200a-4c7b-abe9-247f22afeadd/manager/0.log" Jul 18 17:27:21.049742 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:21.049665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jvx76_77c17dfc-d6e9-4680-892b-e0fb9075bd89/limitador/0.log" Jul 18 17:27:22.857067 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.856973 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/alertmanager/0.log" Jul 18 17:27:22.882059 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.882032 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/config-reloader/0.log" Jul 18 17:27:22.904644 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.904614 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/kube-rbac-proxy-web/0.log" Jul 18 17:27:22.928491 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.928469 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/kube-rbac-proxy/0.log" Jul 18 17:27:22.950984 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.950950 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/kube-rbac-proxy-metric/0.log" Jul 18 17:27:22.972341 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.972319 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/prom-label-proxy/0.log" Jul 18 17:27:22.999173 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:22.999152 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7b635e28-c576-4148-857f-08c7c68283c8/init-config-reloader/0.log" Jul 18 17:27:23.374298 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.374271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlbmx_03f0e8a5-651a-4464-9dd4-f725eda6bd5c/node-exporter/0.log" Jul 18 17:27:23.400806 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.400781 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlbmx_03f0e8a5-651a-4464-9dd4-f725eda6bd5c/kube-rbac-proxy/0.log" Jul 18 17:27:23.424669 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.424644 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlbmx_03f0e8a5-651a-4464-9dd4-f725eda6bd5c/init-textfile/0.log" Jul 18 17:27:23.794722 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.794699 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cc76df989-n6rv8_dc046995-5a1e-4777-9233-b7c413319c52/telemeter-client/0.log" Jul 18 17:27:23.815861 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.815838 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cc76df989-n6rv8_dc046995-5a1e-4777-9233-b7c413319c52/reload/0.log" Jul 18 17:27:23.838020 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.838001 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cc76df989-n6rv8_dc046995-5a1e-4777-9233-b7c413319c52/kube-rbac-proxy/0.log" Jul 18 17:27:23.869478 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.869457 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/thanos-query/0.log" Jul 18 17:27:23.897967 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.897946 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/kube-rbac-proxy-web/0.log" Jul 18 17:27:23.920081 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.920065 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/kube-rbac-proxy/0.log" Jul 18 17:27:23.953478 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.953451 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/prom-label-proxy/0.log" Jul 18 17:27:23.978251 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:23.978234 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/kube-rbac-proxy-rules/0.log" Jul 18 17:27:24.000769 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.000752 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7954b47d79-tl6wz_a0ba530b-cd9c-4a7c-b080-2c4d8a6a2417/kube-rbac-proxy-metrics/0.log" Jul 18 17:27:24.893105 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893075 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t"] Jul 18 17:27:24.893557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893445 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" Jul 18 17:27:24.893557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893456 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" Jul 18 17:27:24.893557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893470 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328c5450-456b-42b2-b886-d6dabc6d7d2c" containerName="revoke-keys" Jul 18 17:27:24.893557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893475 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="328c5450-456b-42b2-b886-d6dabc6d7d2c" containerName="revoke-keys" Jul 18 17:27:24.893557 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893557 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" Jul 18 17:27:24.893762 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893566 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="47eb8b55-0ee2-48a4-810e-4b8776853719" containerName="istio-proxy" Jul 18 17:27:24.893762 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.893572 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="328c5450-456b-42b2-b886-d6dabc6d7d2c" containerName="revoke-keys" Jul 18 17:27:24.896647 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.896626 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:24.899272 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.899255 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fc66r\"/\"kube-root-ca.crt\"" Jul 18 17:27:24.900256 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.900237 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fc66r\"/\"openshift-service-ca.crt\"" Jul 18 17:27:24.900334 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.900253 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fc66r\"/\"default-dockercfg-sttkp\"" Jul 18 17:27:24.907792 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.907772 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t"] Jul 18 17:27:24.964468 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.964440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-podres\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:24.964653 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.964477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc92v\" (UniqueName: \"kubernetes.io/projected/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-kube-api-access-hc92v\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:24.964653 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.964500 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-proc\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:24.964653 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.964579 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-sys\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:24.964653 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:24.964614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-lib-modules\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065772 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065734 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-sys\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-lib-modules\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-podres\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065800 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-sys\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc92v\" (UniqueName: \"kubernetes.io/projected/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-kube-api-access-hc92v\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065896 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-lib-modules\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.065966 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065910 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-proc\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.066207 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065977 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-podres\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.066207 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.065991 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-proc\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.073912 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.073883 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc92v\" (UniqueName: \"kubernetes.io/projected/2044d4b0-9a5a-4a0f-a66f-1a767ea32fae-kube-api-access-hc92v\") pod \"perf-node-gather-daemonset-vk62t\" (UID: \"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae\") " pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.206244 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.206161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.337448 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.337420 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t"] Jul 18 17:27:25.339622 ip-10-0-139-1 kubenswrapper[2567]: W0718 17:27:25.339593 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2044d4b0_9a5a_4a0f_a66f_1a767ea32fae.slice/crio-680ad65c9ed155683cae6eac0d36ab61b674c399f5905ad57112bda7fa436e85 WatchSource:0}: Error finding container 680ad65c9ed155683cae6eac0d36ab61b674c399f5905ad57112bda7fa436e85: Status 404 returned error can't find the container with id 680ad65c9ed155683cae6eac0d36ab61b674c399f5905ad57112bda7fa436e85 Jul 18 17:27:25.960212 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.960178 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" event={"ID":"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae","Type":"ContainerStarted","Data":"02c4bd95354b5d8bd2a019e68270f8a2ed45130e3cd513d6be9b80ff61d1ecae"} Jul 18 17:27:25.960623 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.960227 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" event={"ID":"2044d4b0-9a5a-4a0f-a66f-1a767ea32fae","Type":"ContainerStarted","Data":"680ad65c9ed155683cae6eac0d36ab61b674c399f5905ad57112bda7fa436e85"} Jul 18 17:27:25.960623 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.960270 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:25.976294 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:25.976247 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" podStartSLOduration=1.976232157 podStartE2EDuration="1.976232157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 17:27:25.974933214 +0000 UTC m=+3051.113695879" watchObservedRunningTime="2026-07-18 17:27:25.976232157 +0000 UTC m=+3051.114994820" Jul 18 17:27:28.096376 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:28.096295 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qmdfl_64bc4b28-2f6c-4b23-93e2-baab4394473c/dns/0.log" Jul 18 17:27:28.123324 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:28.123298 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qmdfl_64bc4b28-2f6c-4b23-93e2-baab4394473c/kube-rbac-proxy/0.log" Jul 18 17:27:28.204262 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:28.204232 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tz5zg_940fc19a-b412-455e-ba83-285f02e11730/dns-node-resolver/0.log" Jul 18 17:27:28.771982 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:28.771958 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4s8cp_5379291e-07ab-4173-83da-9ee82abd6e42/node-ca/0.log" Jul 18 17:27:29.751259 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:29.751225 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rpn86_50c0faeb-95d1-489b-b792-e67b37b5d0c6/istio-proxy/0.log" Jul 18 17:27:29.874737 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:29.874699 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6whd6_8d1e349e-3ecd-46d1-beda-6c7a76d82894/discovery/0.log" Jul 18 17:27:29.897408 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:29.897386 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c475ff8c-bv4fz_932ff878-af0b-4930-b433-b066253f1e87/kube-auth-proxy/0.log" Jul 18 17:27:29.950782 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:29.950757 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/0.log" Jul 18 17:27:29.972734 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:29.972706 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/1.log" Jul 18 17:27:30.796298 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:30.796271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fzrhq_044fcfca-f17d-4ee2-887c-add500469cdf/serve-healthcheck-canary/0.log" Jul 18 17:27:31.528723 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:31.528698 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4p87_4fa6655f-6638-4835-b15e-c1965a3eefd4/kube-rbac-proxy/0.log" Jul 18 17:27:31.554890 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:31.554866 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4p87_4fa6655f-6638-4835-b15e-c1965a3eefd4/exporter/0.log" Jul 18 17:27:31.579633 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:31.579611 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4p87_4fa6655f-6638-4835-b15e-c1965a3eefd4/extractor/0.log" Jul 18 17:27:31.976002 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:31.975973 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fc66r/perf-node-gather-daemonset-vk62t" Jul 18 17:27:33.832165 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:33.832141 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ghnr4_aeb58dd5-641b-4d4b-9da9-52f4f543d061/manager/0.log" Jul 18 17:27:34.060235 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:34.060134 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rjw9z_cdd95720-ff0e-4bbb-86e3-18e356908c76/manager/0.log" Jul 18 17:27:34.274587 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:34.274560 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-697w9_805b14f7-698f-43a7-8ef3-31b15d16dc3d/server/0.log" Jul 18 17:27:34.591875 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:34.591848 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c768ddb5d-wp5gw_08e06ba3-2865-4762-a85e-c2f0906bed81/manager/0.log" Jul 18 17:27:34.612242 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:34.612217 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c768ddb5d-wp5gw_08e06ba3-2865-4762-a85e-c2f0906bed81/copy-manifests/0.log" Jul 18 17:27:35.494942 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.494915 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rpn86_50c0faeb-95d1-489b-b792-e67b37b5d0c6/istio-proxy/0.log" Jul 18 17:27:35.611010 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.610988 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6whd6_8d1e349e-3ecd-46d1-beda-6c7a76d82894/discovery/0.log" Jul 18 17:27:35.624639 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.624617 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c475ff8c-bv4fz_932ff878-af0b-4930-b433-b066253f1e87/kube-auth-proxy/0.log" Jul 18 17:27:35.659339 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.659310 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/0.log" Jul 18 17:27:35.684248 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.684224 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/1.log" Jul 18 17:27:35.953449 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:35.953422 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qw5jb_44ba7eee-05d9-45e2-82fc-e742620b09a7/openshift-lws-operator/0.log" Jul 18 17:27:36.959572 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:36.959532 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rpn86_50c0faeb-95d1-489b-b792-e67b37b5d0c6/istio-proxy/0.log" Jul 18 17:27:37.081436 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:37.081406 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6whd6_8d1e349e-3ecd-46d1-beda-6c7a76d82894/discovery/0.log" Jul 18 17:27:37.093332 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:37.093306 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c475ff8c-bv4fz_932ff878-af0b-4930-b433-b066253f1e87/kube-auth-proxy/0.log" Jul 18 17:27:37.158678 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:37.158648 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/0.log" Jul 18 17:27:37.182690 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:37.182663 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-f6v4s_58629c39-717d-4869-b1d2-b21f641450f7/istio-proxy/1.log" Jul 18 17:27:42.356021 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:42.355996 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hswxz_d9291fc5-ce9c-4fec-a60a-27ec7a2ba957/kube-multus-additional-cni-plugins/0.log" Jul 18 17:27:42.378262 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:42.378233 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hswxz_d9291fc5-ce9c-4fec-a60a-27ec7a2ba957/egress-router-binary-copy/0.log" Jul 18 17:27:42.402153 ip-10-0-139-1 kubenswrapper[2567]: I0718 17:27:42.402119 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hswxz_d9291fc5-ce9c-4fec-a60a-27ec7a2ba957/cni-plugins/0.log"