Jul 08 21:06:04.301587 ip-10-0-128-213 systemd[1]: Starting Kubernetes Kubelet... Jul 08 21:06:04.716275 ip-10-0-128-213 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:06:04.716275 ip-10-0-128-213 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 21:06:04.716275 ip-10-0-128-213 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:06:04.716275 ip-10-0-128-213 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 21:06:04.716275 ip-10-0-128-213 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:06:04.717727 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.717640 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 21:06:04.721860 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721845 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.721860 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721860 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721864 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721867 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721870 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721873 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721876 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721879 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721882 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721884 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721887 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721890 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721892 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721895 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721897 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721900 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721905 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721910 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721913 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721916 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.721924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721919 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721922 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721925 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721928 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721930 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721933 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721937 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721942 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721946 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721953 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721956 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721958 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721961 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721964 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721966 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721969 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721971 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721974 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721976 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.722449 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721979 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721981 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721984 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721986 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721989 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721991 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721994 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721996 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.721998 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722001 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722004 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722006 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722008 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722011 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722014 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722018 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722020 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722023 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722026 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722028 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.723163 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722031 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722034 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722036 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722038 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722041 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722043 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722046 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722048 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722050 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722053 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722055 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722058 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722060 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722062 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722065 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722068 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722071 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722074 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722076 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722078 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.723659 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722081 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722083 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722085 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722088 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722091 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722093 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722097 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722100 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722103 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722105 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722107 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722110 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722113 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722116 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722120 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722123 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722126 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722128 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.724116 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.722131 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.724988 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.724972 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.724988 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.724987 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.724991 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.724995 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.724999 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725001 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725004 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725008 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725012 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725015 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725018 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725021 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725023 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725026 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725029 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725032 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725034 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725037 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725040 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725043 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725046 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.725041 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725048 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725051 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725054 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725057 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725060 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725062 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725065 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725067 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725077 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725080 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725082 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725085 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725087 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725094 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725097 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725100 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725102 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725105 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725107 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.725605 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725111 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725114 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725117 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725120 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725122 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725125 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725127 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725130 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725132 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725134 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725137 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725140 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725142 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725145 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725147 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725150 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725153 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725156 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725158 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725160 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.726062 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725164 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725166 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725169 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725171 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725174 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725177 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725179 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725182 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725184 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725187 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725189 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725191 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725194 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725196 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725198 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725201 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725203 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725206 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725208 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725211 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.726553 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725213 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725215 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725218 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725221 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725224 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725226 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725229 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725232 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725250 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725254 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725256 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725259 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725262 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725266 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725270 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725273 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725276 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725279 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725281 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.727034 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725681 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725687 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725690 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725693 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725696 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725699 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725701 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725704 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725707 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725710 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725713 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725716 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725718 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725721 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725723 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725727 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725732 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725734 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725737 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725740 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.727513 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725743 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725745 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725748 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725751 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725754 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725756 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725759 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725761 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725764 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725782 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725787 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725790 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725793 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725796 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725798 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725801 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725804 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725806 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725809 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.728016 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725812 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725816 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725819 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725822 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725825 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725827 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725830 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725832 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725835 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725838 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725841 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725844 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725847 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725850 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725853 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725857 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725859 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725862 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725865 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725867 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.728469 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725870 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725872 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725875 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725877 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725880 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725882 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725884 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725887 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725889 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725892 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725894 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725897 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725899 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725903 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725905 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725908 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725910 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725913 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725915 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.728942 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725919 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725924 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725926 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725929 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725933 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725936 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725939 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725941 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725943 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725946 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725949 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725951 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725953 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725956 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725958 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725961 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725963 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725965 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725968 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.729442 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725971 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.725973 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726036 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726042 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726047 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726051 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726055 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726058 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726062 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726066 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726069 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726072 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726076 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726079 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726082 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726085 2565 flags.go:64] FLAG: --cgroup-root="" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726088 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726091 2565 flags.go:64] FLAG: --client-ca-file="" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726095 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726098 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726102 2565 flags.go:64] FLAG: --cluster-domain="" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726105 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726108 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726111 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 21:06:04.729893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726114 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726118 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726121 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726124 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726127 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726130 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726133 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726135 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726138 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726141 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726145 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726148 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726150 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726153 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726156 2565 flags.go:64] FLAG: --enable-server="true" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726159 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726167 2565 flags.go:64] FLAG: --event-burst="100" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726170 2565 flags.go:64] FLAG: --event-qps="50" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726173 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726176 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726178 2565 flags.go:64] FLAG: --eviction-hard="" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726182 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726185 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726188 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726190 2565 flags.go:64] FLAG: --eviction-soft="" Jul 08 21:06:04.730476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726193 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726196 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726200 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726202 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726205 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726208 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726211 2565 flags.go:64] FLAG: --feature-gates="" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726215 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726218 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726221 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726224 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726227 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726230 2565 flags.go:64] FLAG: --help="false" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726232 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-128-213.ec2.internal" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726252 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726256 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726259 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726262 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726265 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726268 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726271 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726274 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726277 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 21:06:04.731112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726280 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726283 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726286 2565 flags.go:64] FLAG: --kube-reserved="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726289 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726292 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726295 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726297 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726300 2565 flags.go:64] FLAG: --lock-file="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726303 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726306 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726308 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726313 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726317 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726320 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726322 2565 flags.go:64] FLAG: --logging-format="text" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726325 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726328 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726331 2565 flags.go:64] FLAG: --manifest-url="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726334 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726337 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726340 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726344 2565 flags.go:64] FLAG: --max-pods="110" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726347 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726350 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726352 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 21:06:04.731690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726355 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726358 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726361 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726364 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726371 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726374 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726377 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726380 2565 flags.go:64] FLAG: --pod-cidr="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726383 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726388 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726391 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726394 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726396 2565 flags.go:64] FLAG: --port="10250" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726399 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726402 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0017490eacc30013e" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726405 2565 flags.go:64] FLAG: --qos-reserved="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726408 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726410 2565 flags.go:64] FLAG: --register-node="true" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726413 2565 flags.go:64] FLAG: --register-with-taints="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726416 2565 flags.go:64] FLAG: --registry-burst="10" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726420 2565 flags.go:64] FLAG: --registry-qps="5" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726423 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726425 2565 flags.go:64] FLAG: --reserved-memory="" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726429 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726432 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 21:06:04.732383 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726434 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726437 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726440 2565 flags.go:64] FLAG: --runonce="false" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726442 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726445 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726448 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726451 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726453 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726456 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726459 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726462 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726464 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726467 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726470 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726474 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726477 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726480 2565 flags.go:64] FLAG: --system-cgroups="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726483 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726489 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726492 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726496 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726507 2565 flags.go:64] FLAG: --tls-min-version="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726510 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726514 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726517 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 21:06:04.733444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726520 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726523 2565 flags.go:64] FLAG: --v="2" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726527 2565 flags.go:64] FLAG: --version="false" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726532 2565 flags.go:64] FLAG: --vmodule="" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726535 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726538 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726670 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726674 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726677 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726680 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726683 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726685 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726688 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726690 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726693 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726695 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726698 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726700 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726703 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726705 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726708 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.734491 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726713 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726715 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726718 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726721 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726723 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726726 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726728 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726731 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726733 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726737 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726740 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726742 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726745 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726747 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726750 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726753 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726757 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726760 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726763 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.735424 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726765 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726768 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726770 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726773 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726775 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726778 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726780 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726783 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726786 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726788 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726791 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726795 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726798 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726803 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726806 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726808 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726811 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726814 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726816 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726818 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.736173 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726821 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726823 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726827 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726829 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726832 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726835 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726837 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726841 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726844 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726866 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726869 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726872 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726876 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726878 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726881 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726883 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726886 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726888 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726891 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726893 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.736815 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726896 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726898 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726901 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726904 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726906 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726910 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726913 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726916 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726919 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726921 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726924 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726926 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726929 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726931 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726935 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726938 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726940 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726943 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726946 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726948 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.737657 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726951 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726953 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726956 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726959 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.726962 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.726969 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.734478 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.734497 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.734522 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.734531 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734613 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734621 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734638 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734646 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.738291 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734653 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734658 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734664 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734669 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734673 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734677 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734682 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734686 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734690 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734695 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734699 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734702 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734708 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734712 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734716 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734720 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734724 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734729 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734733 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734737 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.738763 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734742 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734746 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734752 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734756 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734760 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734766 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734771 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734775 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734779 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734784 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734788 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734793 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734799 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734805 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734810 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734815 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734820 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734824 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734829 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734834 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.739391 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734838 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734842 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734846 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734850 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734855 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734859 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734863 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734866 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734870 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734875 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734879 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734883 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734887 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734892 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734896 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734900 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734905 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734909 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734915 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734919 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.739923 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734923 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734927 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734931 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734935 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734940 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734944 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734947 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734952 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734956 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734960 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734965 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734969 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734973 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734977 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734981 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734985 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734989 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734993 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.734997 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.740422 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735001 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735006 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735010 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735014 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735018 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735022 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735026 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735030 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735034 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735038 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735042 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735047 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735052 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735056 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735060 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.740924 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735064 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.735072 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735261 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735269 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735274 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735278 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735282 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735287 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735293 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735297 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735301 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735305 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735310 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735314 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735318 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:06:04.741343 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735321 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735325 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735329 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735333 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735337 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735341 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735345 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735349 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735353 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735360 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735365 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735370 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735375 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735379 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735383 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735388 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735393 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735397 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735401 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735405 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:06:04.741703 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735409 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735413 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735417 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735421 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735425 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735429 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735433 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735437 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735441 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735445 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735449 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735453 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735458 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735462 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735466 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735470 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735474 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735478 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735483 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:06:04.742213 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735487 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735491 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735494 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735499 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735503 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735508 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735512 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735516 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735520 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735525 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735529 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735533 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735537 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735540 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735545 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735549 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735553 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735557 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735561 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735565 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:06:04.743004 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735569 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735573 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735577 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735580 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735585 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735590 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735595 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735602 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735608 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735612 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735616 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735620 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735625 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735629 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735633 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735637 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735642 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735647 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735652 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:06:04.743848 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735656 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735660 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735664 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735669 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735673 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735677 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735681 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:04.735685 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.735693 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.735878 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.738469 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.739806 2565 server.go:1019] "Starting client certificate rotation" Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.739907 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.739940 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:06:04.744628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.742788 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:06:04.745111 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.744448 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:06:04.748423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.748405 2565 log.go:25] "Validated CRI v1 runtime API" Jul 08 21:06:04.752104 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.752090 2565 log.go:25] "Validated CRI v1 image API" Jul 08 21:06:04.752826 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.752811 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 21:06:04.754912 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.754885 2565 fs.go:135] Filesystem UUIDs: map[4a2a6c58-c91b-470a-b713-7628da291ca5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fe6ccd29-3bc5-4cf3-8074-02162b45a3fb:/dev/nvme0n1p3] Jul 08 21:06:04.754978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.754910 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 21:06:04.761421 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.761310 2565 manager.go:216] Machine: {Timestamp:2026-07-08 21:06:04.758411509 +0000 UTC m=+0.330156358 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100686 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec272147f907a42a3a3abfa645980a54 SystemUUID:ec272147-f907-a42a-3a3a-bfa645980a54 BootID:d4a18cde-3840-4774-b737-cd5e9b1e7c0e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b9:46:3a:78:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b9:46:3a:78:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:9c:ec:ad:89:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 21:06:04.761421 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.761415 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 21:06:04.761539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.761510 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 21:06:04.764073 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.764048 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 21:06:04.764202 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.764075 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 21:06:04.765580 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.765569 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 21:06:04.765580 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.765580 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 21:06:04.765655 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.765587 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:06:04.767411 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.767396 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:06:04.767952 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.767941 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:06:04.769126 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.769116 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 21:06:04.772044 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772032 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:06:04.772144 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772135 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 21:06:04.772182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772166 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 08 21:06:04.772182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772175 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 21:06:04.772245 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772187 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 21:06:04.772245 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772196 2565 kubelet.go:398] "Adding apiserver pod source" Jul 08 21:06:04.772245 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772205 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 21:06:04.772559 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772547 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:06:04.772618 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.772597 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:06:04.774671 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.774653 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 21:06:04.777457 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.777443 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 21:06:04.777516 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.777476 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 21:06:04.778697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778686 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778703 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778710 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778715 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778723 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778729 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778736 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778741 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778747 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 21:06:04.778748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778752 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 21:06:04.778965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778769 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 21:06:04.778965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.778774 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 21:06:04.780412 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.780399 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 21:06:04.780462 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.780416 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 21:06:04.780694 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.780673 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 21:06:04.780762 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.780691 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 21:06:04.783414 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.783394 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 21:06:04.783414 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.783402 2565 server.go:1268] "Started kubelet" Jul 08 21:06:04.783534 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.783434 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 21:06:04.783534 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.783473 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 21:06:04.783534 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.783468 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 21:06:04.784114 ip-10-0-128-213 systemd[1]: Started Kubernetes Kubelet. Jul 08 21:06:04.785927 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.785914 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 08 21:06:04.787587 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.787568 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n99rp" Jul 08 21:06:04.787712 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.787700 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 21:06:04.788335 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.788316 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 21:06:04.790581 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.789350 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-213.ec2.internal.18c06c8391ebcff8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-213.ec2.internal,UID:ip-10-0-128-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-213.ec2.internal,},FirstTimestamp:2026-07-08 21:06:04.78338252 +0000 UTC m=+0.355127371,LastTimestamp:2026-07-08 21:06:04.78338252 +0000 UTC m=+0.355127371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-213.ec2.internal,}" Jul 08 21:06:04.792451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.792434 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n99rp" Jul 08 21:06:04.793483 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.793465 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 21:06:04.793483 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.793468 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 21:06:04.793619 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.793546 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:04.794326 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.794168 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 21:06:04.794434 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.794228 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 21:06:04.794500 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.794438 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 21:06:04.795011 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.794990 2565 factory.go:55] Registering systemd factory Jul 08 21:06:04.795011 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795011 2565 factory.go:223] Registration of the systemd container factory successfully Jul 08 21:06:04.795451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795432 2565 factory.go:153] Registering CRI-O factory Jul 08 21:06:04.795451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795452 2565 factory.go:223] Registration of the crio container factory successfully Jul 08 21:06:04.795579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795502 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 21:06:04.795579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795527 2565 factory.go:103] Registering Raw factory Jul 08 21:06:04.795579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795543 2565 manager.go:1196] Started watching for new ooms in manager Jul 08 21:06:04.795579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795556 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 08 21:06:04.795579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.795565 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 08 21:06:04.796157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.796135 2565 manager.go:349] Starting recovery of all containers Jul 08 21:06:04.798420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.798399 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:04.802717 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.802695 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-213.ec2.internal\" not found" node="ip-10-0-128-213.ec2.internal" Jul 08 21:06:04.808236 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.808215 2565 manager.go:354] Recovery completed Jul 08 21:06:04.812207 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.812193 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:04.813913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.813895 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:04.814026 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.813926 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:04.814026 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.813939 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:04.814425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.814411 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 21:06:04.814425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.814419 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:04.814425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.814425 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 21:06:04.814591 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.814445 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:06:04.815067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815053 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:04.815124 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815079 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:04.815124 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815092 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:04.815648 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815633 2565 policy_none.go:49] "None policy: Start" Jul 08 21:06:04.815714 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815653 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 21:06:04.815714 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.815667 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 21:06:04.816652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.816638 2565 policy_none.go:47] "Start" Jul 08 21:06:04.852941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.852832 2565 manager.go:337] "Starting Device Plugin manager" Jul 08 21:06:04.853040 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.852971 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 21:06:04.853040 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.852986 2565 server.go:87] "Starting device plugin registration server" Jul 08 21:06:04.853221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.853207 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 21:06:04.853305 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.853222 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 21:06:04.853371 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.853317 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 21:06:04.853436 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.853401 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 21:06:04.853436 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.853427 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 21:06:04.853903 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.853881 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 21:06:04.853979 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.853932 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:04.941948 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.941923 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 21:06:04.943281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.943135 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 21:06:04.943281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.943165 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 21:06:04.943281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.943185 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 21:06:04.943281 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.943215 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 21:06:04.945523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.945507 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:04.953359 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.953346 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:04.954098 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.954083 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:04.954189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.954111 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:04.954189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.954121 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:04.954189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.954152 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-213.ec2.internal" Jul 08 21:06:04.962291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:04.962277 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-213.ec2.internal" Jul 08 21:06:04.962367 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.962296 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-213.ec2.internal\": node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:04.982995 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:04.982940 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.046368 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.046345 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"] Jul 08 21:06:05.046431 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.046401 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:05.047146 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.047131 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:05.047288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.047157 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:05.047288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.047167 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:05.048513 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.048501 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:05.048637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.048625 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.048679 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.048663 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:05.049161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049146 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:05.049229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049146 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:05.049229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049175 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:05.049229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049188 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:05.049229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049197 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:05.049229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.049210 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:05.050888 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.050874 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.050960 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.050903 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:06:05.052366 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.052344 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:06:05.052475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.052376 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:06:05.052475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.052401 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:06:05.076916 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.076898 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-213.ec2.internal\" not found" node="ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.080372 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.080356 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-213.ec2.internal\" not found" node="ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.083359 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.083334 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.097242 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.097224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d32b06a52fe2c12e89c7b42a8cc8f0b1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"d32b06a52fe2c12e89c7b42a8cc8f0b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.097298 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.097261 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.097298 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.097280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.184008 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.183990 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.198439 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.198531 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.198568 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d32b06a52fe2c12e89c7b42a8cc8f0b1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"d32b06a52fe2c12e89c7b42a8cc8f0b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.198568 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.198631 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198491 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/683549c5cb15901e8f0af8813277f06a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"683549c5cb15901e8f0af8813277f06a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.198631 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.198618 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d32b06a52fe2c12e89c7b42a8cc8f0b1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"d32b06a52fe2c12e89c7b42a8cc8f0b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.284265 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.284219 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.386482 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.386464 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.386556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.386501 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.386599 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.386575 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.486599 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.486571 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.587788 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.587715 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found" Jul 08 21:06:05.686217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.686195 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:05.693838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.693814 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.703920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.703898 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:06:05.705974 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.705961 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" Jul 08 21:06:05.715399 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.715382 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:06:05.740401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.740374 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 21:06:05.740776 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.740481 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:06:05.740776 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.740511 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:06:05.740776 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.740525 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:06:05.774189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.774170 2565 apiserver.go:52] "Watching apiserver" Jul 08 21:06:05.793793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.793775 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 21:06:05.793882 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.793781 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 21:06:05.795295 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.795273 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 21:01:05 +0000 UTC" deadline="2028-01-14 15:04:29.605085332 +0000 UTC" Jul 08 21:06:05.795404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.795299 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13313h58m23.809790718s" Jul 08 21:06:05.795404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.795279 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-rz5lw","openshift-network-operator/iptables-alerter-6j4zm","openshift-ovn-kubernetes/ovnkube-node-6vl4t","kube-system/konnectivity-agent-ckdxv","kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849","openshift-cluster-node-tuning-operator/tuned-zfzgm","openshift-image-registry/node-ca-nl9b5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal","openshift-multus/multus-hqhtz","openshift-network-diagnostics/network-check-target-l5tkm","kube-system/global-pull-secret-syncer-8cgls","openshift-dns/node-resolver-j4ccx","openshift-multus/multus-additional-cni-plugins-9cb7q"] Jul 08 21:06:05.797066 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.797047 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.798282 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.798262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:05.798389 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.798341 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:05.799414 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.799397 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.799579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.799559 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jns75\"" Jul 08 21:06:05.799929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.799914 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.800009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.799915 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.800709 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.800697 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:05.801674 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.801656 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-27rm9\"" Jul 08 21:06:05.801770 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.801753 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 21:06:05.801908 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.801895 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.802768 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.802749 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 21:06:05.802861 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.802746 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 21:06:05.802861 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.802804 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.802861 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.802816 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 21:06:05.802861 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.802808 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.803119 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.803103 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.803180 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.803145 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 21:06:05.803180 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.803165 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t46bn\"" Jul 08 21:06:05.803433 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.803416 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 21:06:05.804056 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804042 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.804116 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 21:06:05.804297 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804285 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 21:06:05.804378 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804357 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:05.804538 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.805168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.804980 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fthh\"" Jul 08 21:06:05.805168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.805081 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 21:06:05.807075 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:05.807735 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807716 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 21:06:05.807843 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807737 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 21:06:05.807843 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807806 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.807954 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807927 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.808011 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807980 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:05.808140 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807980 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dbk4q\"" Jul 08 21:06:05.808771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.807979 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:06:05.808866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.808799 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dchgh\"" Jul 08 21:06:05.808931 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.808883 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.809120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.809102 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.809687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.809658 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.809797 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.809668 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.809887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.809818 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-984pn\"" Jul 08 21:06:05.809887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.809831 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.810312 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.810296 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:06:05.810493 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.810478 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 21:06:05.810607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.810589 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 21:06:05.811041 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.811025 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vghwf\"" Jul 08 21:06:05.811329 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.811302 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:05.811407 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.811378 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:05.812029 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.812013 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8447c\"" Jul 08 21:06:05.812238 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.812224 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 21:06:05.812526 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.812512 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:05.812577 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:05.812566 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:05.825067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.825051 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnwq7" Jul 08 21:06:05.834551 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.834534 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnwq7" Jul 08 21:06:05.864400 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.864379 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:05.895798 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.895781 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 21:06:05.902822 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902795 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-sys-fs\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.902923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902826 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-serviceca\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:05.902923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-socket-dir-parent\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.902923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-multus\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-config\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.902968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.903072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-hosts-file\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:05.903072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-multus-certs\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-cnibin\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903093 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-var-lib-kubelet\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903153 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9cr\" (UniqueName: \"kubernetes.io/projected/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-kube-api-access-cd9cr\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-bin\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903205 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-env-overrides\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903231 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7df\" (UniqueName: \"kubernetes.io/projected/67b84fff-9c89-46fa-b87f-cddd57426495-kube-api-access-7r7df\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05078ba8-668e-4f03-88ec-75ba7ad49383-agent-certs\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-sys\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903334 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-host\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4gp4\" (UniqueName: \"kubernetes.io/projected/6a1ad78f-6098-41e9-b9d8-6483fb351125-kube-api-access-n4gp4\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-systemd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903447 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-var-lib-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysconfig\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-system-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903522 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-os-release\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-socket-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-device-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903639 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-daemon-config\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903721 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pnlb\" (UniqueName: \"kubernetes.io/projected/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-kube-api-access-2pnlb\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-tuned\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-tmp-dir\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903790 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-dbus\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:05.903863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903859 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-log-socket\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-k8s-cni-cncf-io\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903909 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-iptables-alerter-script\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903949 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-etc-kubernetes\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.903985 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-systemd-units\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904008 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-systemd\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-lib-modules\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kn6\" (UniqueName: \"kubernetes.io/projected/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-kube-api-access-m6kn6\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-conf-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-slash\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-netns\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904258 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05078ba8-668e-4f03-88ec-75ba7ad49383-konnectivity-ca\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.904448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-etc-selinux\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904337 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-cni-binary-copy\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-kubelet-config\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-ovn\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-script-lib\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj74z\" (UniqueName: \"kubernetes.io/projected/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-kube-api-access-rj74z\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904476 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-conf\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904491 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-registration-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-hostroot\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904579 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-netd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904601 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-modprobe-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-kubernetes\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjppp\" (UniqueName: \"kubernetes.io/projected/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-kube-api-access-fjppp\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904708 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-netns\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904738 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-run\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-tmp\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-etc-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904853 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-system-cni-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-os-release\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcddr\" (UniqueName: \"kubernetes.io/projected/60ecf469-6188-4015-9553-54f322b6bfe8-kube-api-access-rcddr\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cz54\" (UniqueName: \"kubernetes.io/projected/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kube-api-access-2cz54\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-bin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.904979 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-node-log\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67b84fff-9c89-46fa-b87f-cddd57426495-ovn-node-metrics-cert\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-host-slash\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-host\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905073 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-cnibin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905097 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-kubelet\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:05.905899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:05.905120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-kubelet\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.005671 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005633 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysconfig\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.005671 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-system-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-os-release\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005755 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-system-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-socket-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005758 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysconfig\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005800 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-device-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-daemon-config\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005867 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-socket-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005763 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-os-release\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005884 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pnlb\" (UniqueName: \"kubernetes.io/projected/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-kube-api-access-2pnlb\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-tuned\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.005932 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-tmp-dir\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005939 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-device-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.005957 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-dbus\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.006011 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006075 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-dbus\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.006099 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:06.506075124 +0000 UTC m=+2.077819985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-log-socket\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-k8s-cni-cncf-io\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006223 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-log-socket\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006092 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-cni-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006234 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.006273 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006282 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-k8s-cni-cncf-io\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006275 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-iptables-alerter-script\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.006602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006328 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-tmp-dir\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.006344 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:06.506323995 +0000 UTC m=+2.078068847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-etc-kubernetes\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006409 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-systemd-units\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006486 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-etc-kubernetes\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-systemd\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006509 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-systemd-units\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-lib-modules\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kn6\" (UniqueName: \"kubernetes.io/projected/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-kube-api-access-m6kn6\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-conf-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006594 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-systemd\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-slash\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006624 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-daemon-config\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-netns\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006668 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-netns\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.007569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-run-ovn-kubernetes\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05078ba8-668e-4f03-88ec-75ba7ad49383-konnectivity-ca\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006802 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-etc-selinux\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006812 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-lib-modules\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006817 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-conf-dir\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-cni-binary-copy\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-kubelet-config\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006882 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-iptables-alerter-script\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-ovn\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006905 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-script-lib\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006969 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-ovn\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.006975 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6193c5e9-3127-4409-9460-90223187c853-kubelet-config\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007005 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007010 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-slash\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.008529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-etc-selinux\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007087 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj74z\" (UniqueName: \"kubernetes.io/projected/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-kube-api-access-rj74z\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007119 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-conf\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-registration-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-hostroot\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007230 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-netd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-modprobe-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007299 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-kubernetes\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjppp\" (UniqueName: \"kubernetes.io/projected/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-kube-api-access-fjppp\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-netns\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007377 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-run\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-tmp\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007418 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-netd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007446 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05078ba8-668e-4f03-88ec-75ba7ad49383-konnectivity-ca\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-etc-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009423 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007520 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-etc-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007523 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-system-cni-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007562 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-system-cni-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007565 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-os-release\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcddr\" (UniqueName: \"kubernetes.io/projected/60ecf469-6188-4015-9553-54f322b6bfe8-kube-api-access-rcddr\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007633 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cz54\" (UniqueName: \"kubernetes.io/projected/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kube-api-access-2cz54\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007638 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007666 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-os-release\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007681 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-script-lib\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-bin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-node-log\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007742 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-modprobe-d\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67b84fff-9c89-46fa-b87f-cddd57426495-ovn-node-metrics-cert\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007746 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-registration-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007782 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-hostroot\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-host-slash\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007794 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-sysctl-conf\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.009987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-host\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007821 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-kubernetes\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007831 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-netns\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-cnibin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007848 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-run\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-kubelet\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007891 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-node-log\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.007898 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-kubelet\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008030 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-bin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008091 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-sys-fs\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008101 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-serviceca\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008151 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-sys-fs\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-socket-dir-parent\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-multus\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008205 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-multus-socket-dir-parent\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008231 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-config\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008247 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-host\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.010528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-hosts-file\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-multus-certs\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-cnibin\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008418 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-var-lib-kubelet\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9cr\" (UniqueName: \"kubernetes.io/projected/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-kube-api-access-cd9cr\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-bin\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-env-overrides\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7df\" (UniqueName: \"kubernetes.io/projected/67b84fff-9c89-46fa-b87f-cddd57426495-kube-api-access-7r7df\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008532 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a1ad78f-6098-41e9-b9d8-6483fb351125-cni-binary-copy\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05078ba8-668e-4f03-88ec-75ba7ad49383-agent-certs\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-sys\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-host\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4gp4\" (UniqueName: \"kubernetes.io/projected/6a1ad78f-6098-41e9-b9d8-6483fb351125-kube-api-access-n4gp4\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-serviceca\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008692 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-systemd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-var-lib-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008725 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-cni-bin\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008749 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-cnibin\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008743 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60ecf469-6188-4015-9553-54f322b6bfe8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008784 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-cni-multus\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008800 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-var-lib-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-run-multus-certs\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008843 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-var-lib-kubelet\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008854 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a1ad78f-6098-41e9-b9d8-6483fb351125-host-var-lib-kubelet\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60ecf469-6188-4015-9553-54f322b6bfe8-cnibin\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.008942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-systemd\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-hosts-file\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009035 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-sys\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009084 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-host-slash\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009128 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-env-overrides\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009160 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-run-openvswitch\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.011673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009183 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-host\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009193 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b84fff-9c89-46fa-b87f-cddd57426495-host-kubelet\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.009476 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67b84fff-9c89-46fa-b87f-cddd57426495-ovnkube-config\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.010125 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-etc-tuned\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.010588 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67b84fff-9c89-46fa-b87f-cddd57426495-ovn-node-metrics-cert\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.011177 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05078ba8-668e-4f03-88ec-75ba7ad49383-agent-certs\") pod \"konnectivity-agent-ckdxv\" (UID: \"05078ba8-668e-4f03-88ec-75ba7ad49383\") " pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:06.012243 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.011332 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-tmp\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.014612 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.014370 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:06.014612 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.014398 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:06.014612 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.014462 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:06.014612 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.014546 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:06.514527504 +0000 UTC m=+2.086272370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:06.015940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.015879 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pnlb\" (UniqueName: \"kubernetes.io/projected/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-kube-api-access-2pnlb\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:06.017151 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.017121 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kn6\" (UniqueName: \"kubernetes.io/projected/ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9-kube-api-access-m6kn6\") pod \"tuned-zfzgm\" (UID: \"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9\") " pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.017709 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.017627 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcddr\" (UniqueName: \"kubernetes.io/projected/60ecf469-6188-4015-9553-54f322b6bfe8-kube-api-access-rcddr\") pod \"multus-additional-cni-plugins-9cb7q\" (UID: \"60ecf469-6188-4015-9553-54f322b6bfe8\") " pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.018073 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.018051 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cz54\" (UniqueName: \"kubernetes.io/projected/3e9e9b1f-3bc6-431a-9c72-5d02fddf1891-kube-api-access-2cz54\") pod \"aws-ebs-csi-driver-node-xc849\" (UID: \"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.018295 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.018268 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj74z\" (UniqueName: \"kubernetes.io/projected/0bd150f0-00a6-4cca-bc39-f77f0b8eba38-kube-api-access-rj74z\") pod \"iptables-alerter-6j4zm\" (UID: \"0bd150f0-00a6-4cca-bc39-f77f0b8eba38\") " pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.018737 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.018707 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjppp\" (UniqueName: \"kubernetes.io/projected/56d4bc64-7000-4b7a-bbd5-ad884d2e7262-kube-api-access-fjppp\") pod \"node-ca-nl9b5\" (UID: \"56d4bc64-7000-4b7a-bbd5-ad884d2e7262\") " pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.019088 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.019041 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9cr\" (UniqueName: \"kubernetes.io/projected/a1c66415-3f68-4a2a-9137-f3c95eb1cd13-kube-api-access-cd9cr\") pod \"node-resolver-j4ccx\" (UID: \"a1c66415-3f68-4a2a-9137-f3c95eb1cd13\") " pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.019088 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.019043 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4gp4\" (UniqueName: \"kubernetes.io/projected/6a1ad78f-6098-41e9-b9d8-6483fb351125-kube-api-access-n4gp4\") pod \"multus-hqhtz\" (UID: \"6a1ad78f-6098-41e9-b9d8-6483fb351125\") " pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.019313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.019297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7df\" (UniqueName: \"kubernetes.io/projected/67b84fff-9c89-46fa-b87f-cddd57426495-kube-api-access-7r7df\") pod \"ovnkube-node-6vl4t\" (UID: \"67b84fff-9c89-46fa-b87f-cddd57426495\") " pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.052959 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.052934 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683549c5cb15901e8f0af8813277f06a.slice/crio-d6e7743782777274f7172ac40620ef02717511a0c7ab60db9ff4eaed4bc44b78 WatchSource:0}: Error finding container d6e7743782777274f7172ac40620ef02717511a0c7ab60db9ff4eaed4bc44b78: Status 404 returned error can't find the container with id d6e7743782777274f7172ac40620ef02717511a0c7ab60db9ff4eaed4bc44b78 Jul 08 21:06:06.053860 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.053840 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32b06a52fe2c12e89c7b42a8cc8f0b1.slice/crio-260ff0cc2eeb6a9ae10fbfc01f329594bffb7ba6fbb7c93caefbf4a72d73d533 WatchSource:0}: Error finding container 260ff0cc2eeb6a9ae10fbfc01f329594bffb7ba6fbb7c93caefbf4a72d73d533: Status 404 returned error can't find the container with id 260ff0cc2eeb6a9ae10fbfc01f329594bffb7ba6fbb7c93caefbf4a72d73d533 Jul 08 21:06:06.061474 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.061459 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:06:06.117461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.117410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" Jul 08 21:06:06.124994 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.124974 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba151c0d_391f_40a3_8d7c_ea5ffb55c3d9.slice/crio-942a712c7294fe33f071cf17d335fafc047bdea394f9909cc0778eece9af298b WatchSource:0}: Error finding container 942a712c7294fe33f071cf17d335fafc047bdea394f9909cc0778eece9af298b: Status 404 returned error can't find the container with id 942a712c7294fe33f071cf17d335fafc047bdea394f9909cc0778eece9af298b Jul 08 21:06:06.135523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.135506 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:06.142723 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.142706 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b84fff_9c89_46fa_b87f_cddd57426495.slice/crio-a2232d06adee9afb5eb42be99edfe46798774e6e293ceb2b1e207f1443deaed8 WatchSource:0}: Error finding container a2232d06adee9afb5eb42be99edfe46798774e6e293ceb2b1e207f1443deaed8: Status 404 returned error can't find the container with id a2232d06adee9afb5eb42be99edfe46798774e6e293ceb2b1e207f1443deaed8 Jul 08 21:06:06.150491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.150471 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:06.154836 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.154803 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" Jul 08 21:06:06.160199 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.160178 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05078ba8_668e_4f03_88ec_75ba7ad49383.slice/crio-3e0c296c882483a3bdf564839aa6c4b7625db0f10c57b5282010d2fe84f43903 WatchSource:0}: Error finding container 3e0c296c882483a3bdf564839aa6c4b7625db0f10c57b5282010d2fe84f43903: Status 404 returned error can't find the container with id 3e0c296c882483a3bdf564839aa6c4b7625db0f10c57b5282010d2fe84f43903 Jul 08 21:06:06.161859 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.161838 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ecf469_6188_4015_9553_54f322b6bfe8.slice/crio-5fad6289577140878306fa83d70d82a0553dfc292e64e87988cc264cf9b436ce WatchSource:0}: Error finding container 5fad6289577140878306fa83d70d82a0553dfc292e64e87988cc264cf9b436ce: Status 404 returned error can't find the container with id 5fad6289577140878306fa83d70d82a0553dfc292e64e87988cc264cf9b436ce Jul 08 21:06:06.172800 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.172781 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" Jul 08 21:06:06.179702 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.179682 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9e9b1f_3bc6_431a_9c72_5d02fddf1891.slice/crio-c0e9d75934b236e88e3d62813850e71b4f2aa1062f46b2d153ac461fd9b5c614 WatchSource:0}: Error finding container c0e9d75934b236e88e3d62813850e71b4f2aa1062f46b2d153ac461fd9b5c614: Status 404 returned error can't find the container with id c0e9d75934b236e88e3d62813850e71b4f2aa1062f46b2d153ac461fd9b5c614 Jul 08 21:06:06.189288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.189272 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl9b5" Jul 08 21:06:06.195886 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.195868 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d4bc64_7000_4b7a_bbd5_ad884d2e7262.slice/crio-fafa656d543433096c8f40d30ecb1fdfbdd32b26f8b3336c8603fb665a8845c7 WatchSource:0}: Error finding container fafa656d543433096c8f40d30ecb1fdfbdd32b26f8b3336c8603fb665a8845c7: Status 404 returned error can't find the container with id fafa656d543433096c8f40d30ecb1fdfbdd32b26f8b3336c8603fb665a8845c7 Jul 08 21:06:06.200860 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.200839 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4ccx" Jul 08 21:06:06.207743 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.207724 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c66415_3f68_4a2a_9137_f3c95eb1cd13.slice/crio-807107c6aceeea0a5de9c5f3fbfb84aa39808798ffed65b9de825a46cae9c57b WatchSource:0}: Error finding container 807107c6aceeea0a5de9c5f3fbfb84aa39808798ffed65b9de825a46cae9c57b: Status 404 returned error can't find the container with id 807107c6aceeea0a5de9c5f3fbfb84aa39808798ffed65b9de825a46cae9c57b Jul 08 21:06:06.218852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.218836 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6j4zm" Jul 08 21:06:06.222994 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.222980 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hqhtz" Jul 08 21:06:06.225519 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.225501 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd150f0_00a6_4cca_bc39_f77f0b8eba38.slice/crio-eb9e6bc2348a421bb41e6758a7d44ac8debda7337e5a6c374b6750ceeabbaf34 WatchSource:0}: Error finding container eb9e6bc2348a421bb41e6758a7d44ac8debda7337e5a6c374b6750ceeabbaf34: Status 404 returned error can't find the container with id eb9e6bc2348a421bb41e6758a7d44ac8debda7337e5a6c374b6750ceeabbaf34 Jul 08 21:06:06.230687 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:06.230665 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1ad78f_6098_41e9_b9d8_6483fb351125.slice/crio-f85cc6f5c97d97468e1984763f10f74553452d8c87ba86a9580a907d323551af WatchSource:0}: Error finding container f85cc6f5c97d97468e1984763f10f74553452d8c87ba86a9580a907d323551af: Status 404 returned error can't find the container with id f85cc6f5c97d97468e1984763f10f74553452d8c87ba86a9580a907d323551af Jul 08 21:06:06.232359 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.232339 2565 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-hqhtz_openshift-multus(6a1ad78f-6098-41e9-b9d8-6483fb351125): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 08 21:06:06.232443 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.232369 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-hqhtz" podUID="6a1ad78f-6098-41e9-b9d8-6483fb351125" Jul 08 21:06:06.512617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.512561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:06.512617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.512590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:06.512751 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.512681 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:06.512751 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.512691 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:06.512751 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.512730 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:07.51271464 +0000 UTC m=+3.084459501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:06.512751 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.512745 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:07.512739303 +0000 UTC m=+3.084484142 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:06.613286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.613244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:06.613409 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.613366 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:06.613409 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.613387 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:06.613409 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.613400 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:06.613573 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:06.613461 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:07.613443742 +0000 UTC m=+3.185188600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:06.615733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.615716 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:06.836682 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.836573 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:01:06 +0000 UTC" deadline="2028-04-10 11:46:41.696895998 +0000 UTC" Jul 08 21:06:06.836682 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.836631 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15398h40m34.860269738s" Jul 08 21:06:06.964429 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.964370 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6j4zm" event={"ID":"0bd150f0-00a6-4cca-bc39-f77f0b8eba38","Type":"ContainerStarted","Data":"eb9e6bc2348a421bb41e6758a7d44ac8debda7337e5a6c374b6750ceeabbaf34"} Jul 08 21:06:06.966947 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.966918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" event={"ID":"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891","Type":"ContainerStarted","Data":"c0e9d75934b236e88e3d62813850e71b4f2aa1062f46b2d153ac461fd9b5c614"} Jul 08 21:06:06.973308 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.973280 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ckdxv" event={"ID":"05078ba8-668e-4f03-88ec-75ba7ad49383","Type":"ContainerStarted","Data":"3e0c296c882483a3bdf564839aa6c4b7625db0f10c57b5282010d2fe84f43903"} Jul 08 21:06:06.978468 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.977788 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"a2232d06adee9afb5eb42be99edfe46798774e6e293ceb2b1e207f1443deaed8"} Jul 08 21:06:06.979161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.979119 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:06.980366 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.980338 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" event={"ID":"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9","Type":"ContainerStarted","Data":"942a712c7294fe33f071cf17d335fafc047bdea394f9909cc0778eece9af298b"} Jul 08 21:06:06.985988 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.985960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" event={"ID":"d32b06a52fe2c12e89c7b42a8cc8f0b1","Type":"ContainerStarted","Data":"260ff0cc2eeb6a9ae10fbfc01f329594bffb7ba6fbb7c93caefbf4a72d73d533"} Jul 08 21:06:06.992501 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.992422 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"683549c5cb15901e8f0af8813277f06a","Type":"ContainerStarted","Data":"d6e7743782777274f7172ac40620ef02717511a0c7ab60db9ff4eaed4bc44b78"} Jul 08 21:06:06.996640 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:06.996572 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4ccx" event={"ID":"a1c66415-3f68-4a2a-9137-f3c95eb1cd13","Type":"ContainerStarted","Data":"807107c6aceeea0a5de9c5f3fbfb84aa39808798ffed65b9de825a46cae9c57b"} Jul 08 21:06:07.010390 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.010362 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl9b5" event={"ID":"56d4bc64-7000-4b7a-bbd5-ad884d2e7262","Type":"ContainerStarted","Data":"fafa656d543433096c8f40d30ecb1fdfbdd32b26f8b3336c8603fb665a8845c7"} Jul 08 21:06:07.018943 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.018916 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerStarted","Data":"5fad6289577140878306fa83d70d82a0553dfc292e64e87988cc264cf9b436ce"} Jul 08 21:06:07.028846 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.028819 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hqhtz" event={"ID":"6a1ad78f-6098-41e9-b9d8-6483fb351125","Type":"ContainerStarted","Data":"f85cc6f5c97d97468e1984763f10f74553452d8c87ba86a9580a907d323551af"} Jul 08 21:06:07.032039 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.031996 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-hqhtz" podUID="6a1ad78f-6098-41e9-b9d8-6483fb351125" Jul 08 21:06:07.520968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.520928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:07.521161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.520983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:07.521161 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.521123 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:07.521285 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.521186 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:09.521167554 +0000 UTC m=+5.092912399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:07.521651 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.521615 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:07.521651 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.521680 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:09.521663386 +0000 UTC m=+5.093408226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:07.622051 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.622013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:07.622198 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.622182 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:07.622288 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.622205 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:07.622288 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.622221 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:07.622426 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.622295 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:09.622258459 +0000 UTC m=+5.194003302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:07.742703 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.742674 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:06:07.837328 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.837277 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:01:06 +0000 UTC" deadline="2028-02-02 06:01:35.417105407 +0000 UTC" Jul 08 21:06:07.837328 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.837314 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13760h55m27.579795356s" Jul 08 21:06:07.944237 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.944209 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:07.944436 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.944386 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:07.944798 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.944781 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:07.944897 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.944877 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:07.944961 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:07.944945 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:07.945050 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:07.945032 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:09.542977 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.542935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:09.543482 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.542991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:09.543482 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.543145 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:09.543482 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.543207 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:13.543189113 +0000 UTC m=+9.114933958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:09.543678 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.543666 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:09.543732 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.543720 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:13.54370261 +0000 UTC m=+9.115447454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:09.643913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.643873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:09.644109 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.644042 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:09.644207 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.644137 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:09.644207 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.644152 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:09.644307 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.644220 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:13.644199933 +0000 UTC m=+9.215944792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:09.944122 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.944009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:09.944122 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.944036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:09.944364 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.944150 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:09.944364 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:09.944180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:09.944364 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.944318 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:09.944516 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:09.944394 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:11.943561 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:11.943529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:11.944054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:11.943529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:11.944054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:11.943538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:11.944054 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:11.943714 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:11.944054 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:11.943814 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:11.944054 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:11.943879 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:13.574950 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.574908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:13.575386 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.574959 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:13.575386 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.575057 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:13.575386 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.575065 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:13.575386 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.575119 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:21.57510346 +0000 UTC m=+17.146848296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:13.575386 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.575137 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:21.575128454 +0000 UTC m=+17.146873294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:13.676097 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.676062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:13.676281 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.676248 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:13.676281 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.676276 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:13.676403 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.676291 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:13.676403 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.676353 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:21.676335834 +0000 UTC m=+17.248080683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:13.944473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.944311 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:13.944473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.944317 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:13.944473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:13.944439 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:13.944473 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.944435 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:13.944733 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.944553 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:13.944733 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:13.944629 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:15.944046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:15.944017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:15.944504 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:15.944017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:15.944504 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:15.944123 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:15.944504 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:15.944216 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:15.944504 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:15.944017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:15.944504 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:15.944338 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:17.944612 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:17.944409 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:17.944966 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:17.944434 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:17.944966 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:17.944663 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:17.944966 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:17.944470 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:17.944966 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:17.944727 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:17.944966 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:17.944817 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:18.067823 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:18.067792 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" event={"ID":"ba151c0d-391f-40a3-8d7c-ea5ffb55c3d9","Type":"ContainerStarted","Data":"c149001b783c0f23a7dd9da9037836cae1df4b2ae73233d4ed7c8f7a636f9152"} Jul 08 21:06:18.069694 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:18.069666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" event={"ID":"d32b06a52fe2c12e89c7b42a8cc8f0b1","Type":"ContainerStarted","Data":"b4291419a91d499e1bcb6004baab5ce6be726bc9246d9605af24eb402d100ecb"} Jul 08 21:06:18.086996 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:18.086946 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zfzgm" podStartSLOduration=1.802405268 podStartE2EDuration="13.086929501s" totalImagesPullingTime="11.284524233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:18.086447752 +0000 UTC m=+13.658192613" watchObservedRunningTime="2026-07-08 21:06:18.086929501 +0000 UTC m=+13.658674372" Jul 08 21:06:18.099392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:18.099345 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" podStartSLOduration=12.099329015 podStartE2EDuration="12.099329015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:18.099295141 +0000 UTC m=+13.671040002" watchObservedRunningTime="2026-07-08 21:06:18.099329015 +0000 UTC m=+13.671073878" Jul 08 21:06:19.073694 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.073625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"683549c5cb15901e8f0af8813277f06a","Type":"ContainerStarted","Data":"d0ae92fb92ad9d66fb9305e55a3d2a3e449b976cd307c7ed79b6ea6b5ed9ebb4"} Jul 08 21:06:19.075661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.075633 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4ccx" event={"ID":"a1c66415-3f68-4a2a-9137-f3c95eb1cd13","Type":"ContainerStarted","Data":"9a8b007d95209d3920b17994e282eaafe026a00e79b2bb8f9f23f2c9972c57fa"} Jul 08 21:06:19.077472 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.077443 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl9b5" event={"ID":"56d4bc64-7000-4b7a-bbd5-ad884d2e7262","Type":"ContainerStarted","Data":"edfc25542a8c590c908f6621a3cfc8635c5c6c2d0071fb393b29eaffd4d2bfbe"} Jul 08 21:06:19.079173 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.079150 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerStarted","Data":"d26a62d10a568fd16ad07dc9a7a3fba25b3476711db65518d85beedb8d354193"} Jul 08 21:06:19.080817 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.080788 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6j4zm" event={"ID":"0bd150f0-00a6-4cca-bc39-f77f0b8eba38","Type":"ContainerStarted","Data":"972685e93e4c68db20aec52cfdbe4687e314260df395d8f54c1166d622e1904c"} Jul 08 21:06:19.082541 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.082515 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" event={"ID":"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891","Type":"ContainerStarted","Data":"dee3fb95347cf456a90cd0665c8176a05b1805897538d1cda8a47ba5116ed105"} Jul 08 21:06:19.084202 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.084180 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ckdxv" event={"ID":"05078ba8-668e-4f03-88ec-75ba7ad49383","Type":"ContainerStarted","Data":"0234f059dc75151d7fff24b29cd3ef69d16c60566d9790f190374b56d4108450"} Jul 08 21:06:19.122162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.122123 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6j4zm" podStartSLOduration=2.946679469 podStartE2EDuration="14.122107082s" totalImagesPullingTime="11.175427613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:19.099951899 +0000 UTC m=+14.671696822" watchObservedRunningTime="2026-07-08 21:06:19.122107082 +0000 UTC m=+14.693851944" Jul 08 21:06:19.134597 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.134558 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nl9b5" podStartSLOduration=2.929318756 podStartE2EDuration="14.134543406s" totalImagesPullingTime="11.20522465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:19.134387454 +0000 UTC m=+14.706132312" watchObservedRunningTime="2026-07-08 21:06:19.134543406 +0000 UTC m=+14.706288269" Jul 08 21:06:19.148034 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.147980 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ckdxv" podStartSLOduration=2.909597333 podStartE2EDuration="14.147972447s" totalImagesPullingTime="11.238375114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:19.147696027 +0000 UTC m=+14.719440887" watchObservedRunningTime="2026-07-08 21:06:19.147972447 +0000 UTC m=+14.719717305" Jul 08 21:06:19.163396 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.163358 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j4ccx" podStartSLOduration=2.972232467 podStartE2EDuration="14.163347751s" totalImagesPullingTime="11.191115284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:19.16299805 +0000 UTC m=+14.734742909" watchObservedRunningTime="2026-07-08 21:06:19.163347751 +0000 UTC m=+14.735092610" Jul 08 21:06:19.544202 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.544170 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:19.544747 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.544727 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:19.943625 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.943377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:19.943774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.943424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:19.943774 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:19.943655 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:19.943774 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:19.943736 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:19.943774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:19.943429 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:19.943983 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:19.943835 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:20.087213 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:20.087184 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:20.087568 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:20.087259 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ckdxv" Jul 08 21:06:21.636716 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.636671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:21.636716 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.636714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:21.637285 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.636815 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:21.637285 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.636819 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:21.637285 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.636872 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret podName:6193c5e9-3127-4409-9460-90223187c853 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:37.636853177 +0000 UTC m=+33.208598017 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret") pod "global-pull-secret-syncer-8cgls" (UID: "6193c5e9-3127-4409-9460-90223187c853") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:06:21.637285 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.636893 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:37.636882258 +0000 UTC m=+33.208627107 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:06:21.737439 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.737413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:21.737600 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.737538 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:06:21.737600 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.737558 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:06:21.737600 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.737568 2565 projected.go:196] Error preparing data for projected volume kube-api-access-pklmc for pod openshift-network-diagnostics/network-check-target-l5tkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:21.737710 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.737618 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc podName:e1f29212-1cf7-4e97-9721-7983bd01d019 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:37.737603853 +0000 UTC m=+33.309348710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pklmc" (UniqueName: "kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc") pod "network-check-target-l5tkm" (UID: "e1f29212-1cf7-4e97-9721-7983bd01d019") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:06:21.943723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.943647 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:21.943878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.943663 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:21.943878 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.943764 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:21.943878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:21.943780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:21.943878 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.943862 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:21.944064 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:21.943944 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:22.930842 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:22.930651 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 21:06:23.094560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.094481 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="d26a62d10a568fd16ad07dc9a7a3fba25b3476711db65518d85beedb8d354193" exitCode=0 Jul 08 21:06:23.094560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.094519 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"d26a62d10a568fd16ad07dc9a7a3fba25b3476711db65518d85beedb8d354193"} Jul 08 21:06:23.096627 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.096586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" event={"ID":"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891","Type":"ContainerStarted","Data":"7514406e561eb9cd58308e19eff4120fab3786c4bb8505ed09fdaeea3285fcb1"} Jul 08 21:06:23.100589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100215 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"7dcc90088332cc7afbfc0beeb681b1a03b702ad9305f4f818de71c571d1f673e"} Jul 08 21:06:23.100589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100345 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"68421b40edd5498a0e1a0ee1f4db517ccc5d7e9373bb2f737750625c3fb044fe"} Jul 08 21:06:23.100589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"69b758cb10daca91a44e46e87243e50155ccaf0890120f50012ac18e7567bcf8"} Jul 08 21:06:23.100589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100532 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"df5d8d4d7dffa82ff7873dd08e2113f6ddfaaaa496def397afe456383e977064"} Jul 08 21:06:23.100810 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100664 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"c542e88acb9a73c2412f1c3befec51acb090c8aaf458cea782ca52335cd5868b"} Jul 08 21:06:23.100885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.100867 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"7539261d9b18ed9e084f16fbbb2cea43c5eab00269a0ca6166299662d7b1aeaf"} Jul 08 21:06:23.104689 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.104668 2565 generic.go:356] "Generic (PLEG): container finished" podID="683549c5cb15901e8f0af8813277f06a" containerID="d0ae92fb92ad9d66fb9305e55a3d2a3e449b976cd307c7ed79b6ea6b5ed9ebb4" exitCode=0 Jul 08 21:06:23.104806 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.104698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"683549c5cb15901e8f0af8813277f06a","Type":"ContainerDied","Data":"d0ae92fb92ad9d66fb9305e55a3d2a3e449b976cd307c7ed79b6ea6b5ed9ebb4"} Jul 08 21:06:23.866592 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.866302 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T21:06:22.93083757Z","UUID":"e641bf23-1dbf-495f-a280-552f22d8ba77","Handler":null,"Name":"","Endpoint":""} Jul 08 21:06:23.868118 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.868099 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 21:06:23.868274 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.868139 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 21:06:23.944301 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.944275 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:23.944649 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.944379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:23.944649 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:23.944403 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:23.944649 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:23.944384 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:23.944649 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:23.944483 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:23.944649 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:23.944591 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:24.109061 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:24.109030 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"683549c5cb15901e8f0af8813277f06a","Type":"ContainerStarted","Data":"6769539db33d6a329fc4a3d288f8d5ec458bab59c7eb23f2b40bfa2217849b4f"} Jul 08 21:06:24.122901 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:24.122826 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" podStartSLOduration=18.122813005 podStartE2EDuration="18.122813005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:24.122184146 +0000 UTC m=+19.693929006" watchObservedRunningTime="2026-07-08 21:06:24.122813005 +0000 UTC m=+19.694557863" Jul 08 21:06:25.115108 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.115059 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" event={"ID":"3e9e9b1f-3bc6-431a-9c72-5d02fddf1891","Type":"ContainerStarted","Data":"233fb9f3516de5e7a0deee134984a91f0e15afb21f1c5ad0ddad6fb4df654548"} Jul 08 21:06:25.118741 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.118704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"f8f37d1d0a37fd28131160335366df8e2b417318ec12b17dd5913f55722e95d5"} Jul 08 21:06:25.132214 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.132172 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xc849" podStartSLOduration=3.277543312 podStartE2EDuration="20.13215953s" totalImagesPullingTime="16.854616218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:25.130599511 +0000 UTC m=+20.702344371" watchObservedRunningTime="2026-07-08 21:06:25.13215953 +0000 UTC m=+20.703904388" Jul 08 21:06:25.943621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.943589 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:25.943829 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.943589 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:25.943829 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:25.943725 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:25.943829 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:25.943594 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:25.943994 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:25.943844 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:25.943994 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:25.943903 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:27.129090 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.128854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" event={"ID":"67b84fff-9c89-46fa-b87f-cddd57426495","Type":"ContainerStarted","Data":"85f2bdd277d3b43025a8985c6b554d383fd2f880aa30a67acc995464401ad824"} Jul 08 21:06:27.129755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.129153 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:27.129755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.129209 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:27.129755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.129277 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:27.149372 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.149304 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:27.149509 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.149438 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:27.155160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.154405 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" podStartSLOduration=6.064287709 podStartE2EDuration="22.154387169s" totalImagesPullingTime="16.09009946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:27.154063708 +0000 UTC m=+22.725808581" watchObservedRunningTime="2026-07-08 21:06:27.154387169 +0000 UTC m=+22.726132030" Jul 08 21:06:27.943815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.943747 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:27.944062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.943874 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:27.944062 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:27.943981 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:27.944062 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:27.943877 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:27.944062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:27.944003 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:27.944227 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:27.944108 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:28.791313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.791279 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8cgls"] Jul 08 21:06:28.791733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.791391 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:28.791733 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:28.791495 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:28.794558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.794338 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5tkm"] Jul 08 21:06:28.794558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.794425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:28.794558 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:28.794514 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:28.795066 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.795016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rz5lw"] Jul 08 21:06:28.795156 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:28.795108 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:28.795262 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:28.795202 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:30.943710 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:30.943680 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:30.944101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:30.943722 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:30.944101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:30.943680 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:30.944101 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:30.943804 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:30.944101 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:30.943939 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:30.944101 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:30.944062 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:32.944546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:32.944467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:32.944546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:32.944473 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:32.945002 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:32.944608 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:32.945063 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:32.945027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:32.945171 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:32.945138 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:32.945230 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:32.945214 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:34.146144 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.145891 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="8abcd2b9addc8229f03ddf4855ae5af87e43994e83ab9af72c14f15adffc8963" exitCode=0 Jul 08 21:06:34.146672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.145957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"8abcd2b9addc8229f03ddf4855ae5af87e43994e83ab9af72c14f15adffc8963"} Jul 08 21:06:34.147948 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.147924 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hqhtz" event={"ID":"6a1ad78f-6098-41e9-b9d8-6483fb351125","Type":"ContainerStarted","Data":"d5da527cc7801c6e4eeaa650f7e3dde337d79ac0b80ec306d1e4a603150a41f5"} Jul 08 21:06:34.182422 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.182375 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hqhtz" podStartSLOduration=1.416167169 podStartE2EDuration="29.182361174s" totalImagesPullingTime="27.766194005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 21:06:34.181823012 +0000 UTC m=+29.753567870" watchObservedRunningTime="2026-07-08 21:06:34.182361174 +0000 UTC m=+29.754106033" Jul 08 21:06:34.945023 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.944994 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:34.945220 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.945111 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:34.945220 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:34.945104 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5tkm" podUID="e1f29212-1cf7-4e97-9721-7983bd01d019" Jul 08 21:06:34.945220 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:34.945128 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:34.945220 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:34.945206 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:06:34.945458 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:34.945307 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8cgls" podUID="6193c5e9-3127-4409-9460-90223187c853" Jul 08 21:06:35.725704 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.725684 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeReady" Jul 08 21:06:35.726363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.725818 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 21:06:35.757140 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.757111 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:06:35.774899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.774880 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5926b"] Jul 08 21:06:35.775025 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.775009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.778713 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.778050 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 21:06:35.778713 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.778056 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-78qbx\"" Jul 08 21:06:35.778713 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.778115 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 21:06:35.778713 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.778476 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 21:06:35.782652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.782635 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 21:06:35.789460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.789443 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p578w"] Jul 08 21:06:35.789581 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.789561 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5926b" Jul 08 21:06:35.792145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.792021 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 21:06:35.792145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.792040 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 21:06:35.792145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.792039 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ltsxq\"" Jul 08 21:06:35.806393 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.806375 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:06:35.806485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.806402 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5926b"] Jul 08 21:06:35.806485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.806422 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p578w"] Jul 08 21:06:35.806592 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.806497 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:35.808941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.808923 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 21:06:35.809037 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.808982 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 21:06:35.809101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.809051 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 21:06:35.809101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.809053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vg9kt\"" Jul 08 21:06:35.903601 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.903510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:35.949058 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg95g\" (UniqueName: \"kubernetes.io/projected/b787178f-18f3-4d58-888d-760a65129311-kube-api-access-qg95g\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:35.949159 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029c9231-733f-43bd-b92c-17b495c2adeb-config-volume\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:35.949159 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:35.949226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b787178f-18f3-4d58-888d-760a65129311-cert\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:35.949226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949316 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949316 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949316 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4svj\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949360 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stgcp\" (UniqueName: \"kubernetes.io/projected/029c9231-733f-43bd-b92c-17b495c2adeb-kube-api-access-stgcp\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:35.949407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949390 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c9231-733f-43bd-b92c-17b495c2adeb-tmp-dir\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:35.949533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.949533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.949430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:35.982155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:35.982117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:36.050086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050091 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4svj\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stgcp\" (UniqueName: \"kubernetes.io/projected/029c9231-733f-43bd-b92c-17b495c2adeb-kube-api-access-stgcp\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.050230 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.050267 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050370 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.050323 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:36.55030338 +0000 UTC m=+32.122048233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:36.050637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c9231-733f-43bd-b92c-17b495c2adeb-tmp-dir\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.050637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.050637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050491 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg95g\" (UniqueName: \"kubernetes.io/projected/b787178f-18f3-4d58-888d-760a65129311-kube-api-access-qg95g\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:36.050637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029c9231-733f-43bd-b92c-17b495c2adeb-config-volume\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.050876 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050730 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c9231-733f-43bd-b92c-17b495c2adeb-tmp-dir\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.050876 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.050876 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.050805 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:36.050876 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.050872 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:06:36.550853195 +0000 UTC m=+32.122598051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:36.051029 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050867 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.051029 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.050806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b787178f-18f3-4d58-888d-760a65129311-cert\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:36.051182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.051065 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.051182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.051089 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029c9231-733f-43bd-b92c-17b495c2adeb-config-volume\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.051515 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.051499 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.054174 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.054147 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.054281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.054154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b787178f-18f3-4d58-888d-760a65129311-cert\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:36.054281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.054153 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.057808 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.057780 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stgcp\" (UniqueName: \"kubernetes.io/projected/029c9231-733f-43bd-b92c-17b495c2adeb-kube-api-access-stgcp\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.057954 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.057930 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.058056 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.058036 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg95g\" (UniqueName: \"kubernetes.io/projected/b787178f-18f3-4d58-888d-760a65129311-kube-api-access-qg95g\") pod \"ingress-canary-p578w\" (UID: \"b787178f-18f3-4d58-888d-760a65129311\") " pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:36.058521 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.058502 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4svj\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.123685 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.123665 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p578w" Jul 08 21:06:36.154735 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.154690 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="c5b67c958eabde1b92ae3257a23c822b9486d3c511fe565fe637d3201e5b142a" exitCode=0 Jul 08 21:06:36.154735 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.154719 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"c5b67c958eabde1b92ae3257a23c822b9486d3c511fe565fe637d3201e5b142a"} Jul 08 21:06:36.279152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.279122 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p578w"] Jul 08 21:06:36.282894 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:36.282864 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb787178f_18f3_4d58_888d_760a65129311.slice/crio-f1e475995c0876fc60f011599e2eee1617389c8848aaf1535955744d0946e11f WatchSource:0}: Error finding container f1e475995c0876fc60f011599e2eee1617389c8848aaf1535955744d0946e11f: Status 404 returned error can't find the container with id f1e475995c0876fc60f011599e2eee1617389c8848aaf1535955744d0946e11f Jul 08 21:06:36.554086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.554066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:36.554181 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.554128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:36.554230 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.554202 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:36.554230 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.554213 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:36.554313 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.554232 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:36.554313 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.554285 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:06:37.554269886 +0000 UTC m=+33.126014727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:36.554313 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:36.554301 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:37.554293624 +0000 UTC m=+33.126038461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:36.946589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.946529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:36.946589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.946546 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:36.946987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.946625 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:36.950127 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950105 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:06:36.950272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950142 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:06:36.950272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950142 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:06:36.950272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950191 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfx9c\"" Jul 08 21:06:36.950272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950108 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:06:36.950272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:36.950201 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fk5xk\"" Jul 08 21:06:37.157931 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.157908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p578w" event={"ID":"b787178f-18f3-4d58-888d-760a65129311","Type":"ContainerStarted","Data":"f1e475995c0876fc60f011599e2eee1617389c8848aaf1535955744d0946e11f"} Jul 08 21:06:37.266470 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.266440 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:37.560978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.560951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:37.561126 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.561011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:37.561126 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.561106 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:37.561126 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.561116 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:37.561275 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.561183 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:06:39.561163408 +0000 UTC m=+35.132908251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:37.561275 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.561123 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:37.561275 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.561234 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:39.561222859 +0000 UTC m=+35.132967697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:37.661554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.661526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:06:37.661675 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.661569 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:37.661739 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.661677 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 21:06:37.661790 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:37.661747 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:07:09.66172652 +0000 UTC m=+65.233471372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : secret "metrics-daemon-secret" not found Jul 08 21:06:37.663999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.663971 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6193c5e9-3127-4409-9460-90223187c853-original-pull-secret\") pod \"global-pull-secret-syncer-8cgls\" (UID: \"6193c5e9-3127-4409-9460-90223187c853\") " pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:37.762012 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.761986 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:37.764145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.764122 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklmc\" (UniqueName: \"kubernetes.io/projected/e1f29212-1cf7-4e97-9721-7983bd01d019-kube-api-access-pklmc\") pod \"network-check-target-l5tkm\" (UID: \"e1f29212-1cf7-4e97-9721-7983bd01d019\") " pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:37.856719 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.856670 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8cgls" Jul 08 21:06:37.862824 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:37.862803 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:38.050305 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.050273 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8cgls"] Jul 08 21:06:38.053534 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:38.053511 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6193c5e9_3127_4409_9460_90223187c853.slice/crio-0eda2877469ce19bfdbfd5ee0bf9adac7062d1aedb44620fea66d3dc49a7fa50 WatchSource:0}: Error finding container 0eda2877469ce19bfdbfd5ee0bf9adac7062d1aedb44620fea66d3dc49a7fa50: Status 404 returned error can't find the container with id 0eda2877469ce19bfdbfd5ee0bf9adac7062d1aedb44620fea66d3dc49a7fa50 Jul 08 21:06:38.059793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.059773 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5tkm"] Jul 08 21:06:38.062118 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:06:38.062096 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f29212_1cf7_4e97_9721_7983bd01d019.slice/crio-5d45b9db55ee9370b7fd999c6f61a6fbf8c6b61c7a7a932874c7fb57a6668a04 WatchSource:0}: Error finding container 5d45b9db55ee9370b7fd999c6f61a6fbf8c6b61c7a7a932874c7fb57a6668a04: Status 404 returned error can't find the container with id 5d45b9db55ee9370b7fd999c6f61a6fbf8c6b61c7a7a932874c7fb57a6668a04 Jul 08 21:06:38.161405 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.161340 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5tkm" event={"ID":"e1f29212-1cf7-4e97-9721-7983bd01d019","Type":"ContainerStarted","Data":"5d45b9db55ee9370b7fd999c6f61a6fbf8c6b61c7a7a932874c7fb57a6668a04"} Jul 08 21:06:38.162650 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.162620 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8cgls" event={"ID":"6193c5e9-3127-4409-9460-90223187c853","Type":"ContainerStarted","Data":"0eda2877469ce19bfdbfd5ee0bf9adac7062d1aedb44620fea66d3dc49a7fa50"} Jul 08 21:06:38.165410 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.165386 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="ad631bf1e2711c3f85782437041f10ba649bc6b88c871ad84e8dd4fb43ee4145" exitCode=0 Jul 08 21:06:38.165502 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:38.165420 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"ad631bf1e2711c3f85782437041f10ba649bc6b88c871ad84e8dd4fb43ee4145"} Jul 08 21:06:39.576185 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:39.576143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:39.576210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:39.576372 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:39.576433 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:06:43.576414182 +0000 UTC m=+39.148159036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:39.576501 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:39.576512 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:39.576657 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:39.576543 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:43.576533276 +0000 UTC m=+39.148278128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:39.680228 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:39.680151 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:39.733342 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:39.733273 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:40.862935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:40.862905 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:40.913210 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:40.913125 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:41.359998 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:41.359964 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:41.461869 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:41.461824 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:42.179535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:42.179478 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p578w" event={"ID":"b787178f-18f3-4d58-888d-760a65129311","Type":"ContainerStarted","Data":"165539e8fadf02e190bbf1743c29f8739e9db6890d40960a674e8de222e5997a"} Jul 08 21:06:42.195468 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:42.195416 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p578w" podStartSLOduration=1.915431742 podStartE2EDuration="7.195397862s" totalImagesPullingTime="5.27996612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:42.194275175 +0000 UTC m=+37.766020038" watchObservedRunningTime="2026-07-08 21:06:42.195397862 +0000 UTC m=+37.767142721" Jul 08 21:06:43.605366 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:43.605328 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:43.605384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:43.605491 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:43.605494 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:43.605518 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:43.605553 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:06:51.605536899 +0000 UTC m=+47.177281737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:43.605792 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:43.605568 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:06:51.605562132 +0000 UTC m=+47.177306969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:44.548675 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:44.548622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:44.599289 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:44.599258 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:46.191552 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.191517 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5tkm" event={"ID":"e1f29212-1cf7-4e97-9721-7983bd01d019","Type":"ContainerStarted","Data":"a53d060168e7ed2a097cf66f7d7d5c079ff7f8e954fc96ab150fc6761468e454"} Jul 08 21:06:46.191922 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.191653 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:06:46.193020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.192987 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8cgls" event={"ID":"6193c5e9-3127-4409-9460-90223187c853","Type":"ContainerStarted","Data":"36b10204cfd2cced54ce5669a5094c2eca1e7ac3c0002fca93a72484b15e4316"} Jul 08 21:06:46.195662 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.195639 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="36579d7e5ad25861106ac2352ff4761dcb151fcc3edca229dc503518638ab3f1" exitCode=0 Jul 08 21:06:46.195748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.195669 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"36579d7e5ad25861106ac2352ff4761dcb151fcc3edca229dc503518638ab3f1"} Jul 08 21:06:46.206833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.206791 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l5tkm" podStartSLOduration=34.045386348 podStartE2EDuration="41.206780399s" totalImagesPullingTime="7.161394051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:46.206351836 +0000 UTC m=+41.778096694" watchObservedRunningTime="2026-07-08 21:06:46.206780399 +0000 UTC m=+41.778525257" Jul 08 21:06:46.239155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:46.239116 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8cgls" podStartSLOduration=34.033097021 podStartE2EDuration="41.239103365s" totalImagesPullingTime="7.206006344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:46.238554663 +0000 UTC m=+41.810299523" watchObservedRunningTime="2026-07-08 21:06:46.239103365 +0000 UTC m=+41.810848225" Jul 08 21:06:47.200837 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:47.200806 2565 generic.go:356] "Generic (PLEG): container finished" podID="60ecf469-6188-4015-9553-54f322b6bfe8" containerID="f802afbe9c842fdf176c34637255aa7d67aee5c2b03ff98e52968511678bdbb5" exitCode=0 Jul 08 21:06:47.201223 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:47.200874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerDied","Data":"f802afbe9c842fdf176c34637255aa7d67aee5c2b03ff98e52968511678bdbb5"} Jul 08 21:06:48.206058 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:48.206025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" event={"ID":"60ecf469-6188-4015-9553-54f322b6bfe8","Type":"ContainerStarted","Data":"f94f614ed981f4f2fc60cc720475b32eb5e60776f9ebcf5a1b5bc88adc43f5e2"} Jul 08 21:06:48.227344 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:48.227304 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9cb7q" podStartSLOduration=11.836825159 podStartE2EDuration="43.227285657s" totalImagesPullingTime="31.390460498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:06:48.225764161 +0000 UTC m=+43.797509021" watchObservedRunningTime="2026-07-08 21:06:48.227285657 +0000 UTC m=+43.799030517" Jul 08 21:06:48.344150 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:48.344127 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:48.444135 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:48.444114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:51.663091 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:51.663061 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:51.663107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:51.663200 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:51.663216 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:51.663217 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:51.663283 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:07:07.663269394 +0000 UTC m=+63.235014244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:06:51.663510 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:06:51.663298 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:07:07.663291038 +0000 UTC m=+63.235035878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:06:52.327575 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:52.327544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:52.375599 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:52.375572 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:56.840968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:56.840935 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:56.943047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:56.943021 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:59.142524 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:59.142494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6vl4t" Jul 08 21:06:59.295666 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:59.295640 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:06:59.344960 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:06:59.344936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:00.737894 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:00.737862 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:00.784446 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:00.784427 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:02.527481 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:02.527442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:02.576636 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:02.576612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:07.665021 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:07.664990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:07.665028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:07.665125 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:07.665130 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:07.665147 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:07.665179 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:07:39.665161491 +0000 UTC m=+95.236906330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:07:07.665448 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:07.665191 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:07:39.665185758 +0000 UTC m=+95.236930594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:07:09.677388 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:09.677354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:07:09.677750 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:09.677458 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 21:07:09.677750 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:09.677518 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:08:13.677503825 +0000 UTC m=+129.249248661 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : secret "metrics-daemon-secret" not found Jul 08 21:07:17.202867 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:17.202840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l5tkm" Jul 08 21:07:17.677826 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:17.677794 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:17.731664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:17.731638 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:07:39.666816 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:39.666785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:07:39.666826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:39.666910 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:39.666915 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:39.666929 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:39.666956 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:08:43.666943368 +0000 UTC m=+159.238688205 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:07:39.667221 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:07:39.666969 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:08:43.666963116 +0000 UTC m=+159.238707953 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:08:13.678145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:13.678105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:08:13.678630 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:13.678273 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 21:08:13.678630 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:13.678343 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs podName:fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3 nodeName:}" failed. No retries permitted until 2026-07-08 21:10:15.678328308 +0000 UTC m=+251.250073144 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs") pod "network-metrics-daemon-rz5lw" (UID: "fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3") : secret "metrics-daemon-secret" not found Jul 08 21:08:38.811907 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:38.811869 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" Jul 08 21:08:38.820021 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:38.819996 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5926b" podUID="029c9231-733f-43bd-b92c-17b495c2adeb" Jul 08 21:08:39.443683 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:39.443662 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5926b" Jul 08 21:08:39.443785 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:39.443662 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:08:39.966432 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:39.966405 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rz5lw" podUID="fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3" Jul 08 21:08:43.668936 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:43.668900 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:43.668967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:43.669027 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:43.669068 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:43.669082 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b6b698b5-54tw8: secret "image-registry-tls" not found Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:43.669087 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls podName:029c9231-733f-43bd-b92c-17b495c2adeb nodeName:}" failed. No retries permitted until 2026-07-08 21:10:45.669074032 +0000 UTC m=+281.240818869 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls") pod "dns-default-5926b" (UID: "029c9231-733f-43bd-b92c-17b495c2adeb") : secret "dns-default-metrics-tls" not found Jul 08 21:08:43.669360 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:08:43.669139 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls podName:43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1 nodeName:}" failed. No retries permitted until 2026-07-08 21:10:45.669123118 +0000 UTC m=+281.240867969 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls") pod "image-registry-69b6b698b5-54tw8" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1") : secret "image-registry-tls" not found Jul 08 21:08:48.465336 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:48.465299 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:08:50.944298 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:08:50.944268 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:09:37.005201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:37.005144 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:09:58.499586 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.499552 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-898wv"] Jul 08 21:09:58.501303 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.501286 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.503530 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.503507 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 21:09:58.503721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.503695 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qg55n\"" Jul 08 21:09:58.503827 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.503760 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 21:09:58.506958 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.506940 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ztfgv"] Jul 08 21:09:58.509017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.509000 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hrmnd"] Jul 08 21:09:58.509108 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.509095 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.510939 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.510923 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959"] Jul 08 21:09:58.511040 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.511026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.512141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.512128 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:09:58.512573 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.512554 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.512672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.512559 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 21:09:58.512819 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.512804 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 21:09:58.513549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.513531 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 21:09:58.513651 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.513589 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 21:09:58.513841 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.513819 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 21:09:58.513841 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.513844 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 21:09:58.513987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.513845 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tlg96\"" Jul 08 21:09:58.514213 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.514199 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-898wv"] Jul 08 21:09:58.514437 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.514421 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p4zgr\"" Jul 08 21:09:58.514535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.514514 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 21:09:58.516204 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.516185 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:09:58.516321 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.516261 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 21:09:58.516878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.516796 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n627s\"" Jul 08 21:09:58.516878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.516829 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 21:09:58.517010 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.516972 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 21:09:58.520643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.520616 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 21:09:58.521164 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.521146 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 21:09:58.528355 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.528299 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ztfgv"] Jul 08 21:09:58.529052 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.529035 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hrmnd"] Jul 08 21:09:58.540483 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.540462 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959"] Jul 08 21:09:58.607003 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.606968 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4"] Jul 08 21:09:58.608662 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.608646 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" Jul 08 21:09:58.610408 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.610385 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk"] Jul 08 21:09:58.611075 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.611054 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xnjsf\"" Jul 08 21:09:58.612319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.612300 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2"] Jul 08 21:09:58.612414 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.612392 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.613957 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.613942 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" Jul 08 21:09:58.614471 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.614457 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 21:09:58.614730 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.614715 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:09:58.614886 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.614868 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cpzst\"" Jul 08 21:09:58.615212 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.615122 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 21:09:58.616793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.616775 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:09:58.616873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.616796 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gxg4t\"" Jul 08 21:09:58.617497 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.617481 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 21:09:58.619904 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.619882 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4"] Jul 08 21:09:58.623432 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.623417 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk"] Jul 08 21:09:58.626899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.626877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2"] Jul 08 21:09:58.635148 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-snapshots\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635270 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635169 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635270 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78fsw\" (UniqueName: \"kubernetes.io/projected/d940ecf2-11d4-4c5e-84de-87745c55deb4-kube-api-access-78fsw\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635390 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d940ecf2-11d4-4c5e-84de-87745c55deb4-serving-cert\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635390 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c0aa75c-8b9a-46d8-86e9-492244a01b55-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.635390 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/549e2d5f-64f0-438b-9d66-ccd547074e9d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.635539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-trusted-ca\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.635539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635463 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9dd\" (UniqueName: \"kubernetes.io/projected/6c0aa75c-8b9a-46d8-86e9-492244a01b55-kube-api-access-gx9dd\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.635539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635494 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c0aa75c-8b9a-46d8-86e9-492244a01b55-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.635668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.635668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-tmp\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.635668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635588 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-config\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.635668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qhmh\" (UniqueName: \"kubernetes.io/projected/5bf1d832-d339-4c2e-a370-aa391743633c-kube-api-access-4qhmh\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.635668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.635633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf1d832-d339-4c2e-a370-aa391743633c-serving-cert\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.735991 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.735970 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c0aa75c-8b9a-46d8-86e9-492244a01b55-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.736077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.736077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-tmp\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736147 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:58.736111 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:09:58.736178 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:58.736160 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:09:59.236147185 +0000 UTC m=+234.807892022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:09:58.736257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736202 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rtx\" (UniqueName: \"kubernetes.io/projected/c5656c5c-b830-41ca-9357-1ef150e63125-kube-api-access-96rtx\") pod \"network-check-source-989f8b8dc-s5zz4\" (UID: \"c5656c5c-b830-41ca-9357-1ef150e63125\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" Jul 08 21:09:58.736315 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.736372 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-config\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.736449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qhmh\" (UniqueName: \"kubernetes.io/projected/5bf1d832-d339-4c2e-a370-aa391743633c-kube-api-access-4qhmh\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.736503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf1d832-d339-4c2e-a370-aa391743633c-serving-cert\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.736556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736500 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-snapshots\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736524 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-tmp\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78fsw\" (UniqueName: \"kubernetes.io/projected/d940ecf2-11d4-4c5e-84de-87745c55deb4-kube-api-access-78fsw\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736605 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lgbp\" (UniqueName: \"kubernetes.io/projected/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-kube-api-access-7lgbp\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.736705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736636 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c0aa75c-8b9a-46d8-86e9-492244a01b55-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.736705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d940ecf2-11d4-4c5e-84de-87745c55deb4-serving-cert\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.736900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c0aa75c-8b9a-46d8-86e9-492244a01b55-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.736900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/549e2d5f-64f0-438b-9d66-ccd547074e9d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.737009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.737009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjfw2\" (UniqueName: \"kubernetes.io/projected/b429128e-1987-4465-a48c-219eaf47bd53-kube-api-access-hjfw2\") pod \"volume-data-source-validator-5d456fdbcd-j5fm2\" (UID: \"b429128e-1987-4465-a48c-219eaf47bd53\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" Jul 08 21:09:58.737009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736956 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-trusted-ca\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.737009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.736985 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9dd\" (UniqueName: \"kubernetes.io/projected/6c0aa75c-8b9a-46d8-86e9-492244a01b55-kube-api-access-gx9dd\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.737208 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.737094 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-config\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.737285 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.737218 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d940ecf2-11d4-4c5e-84de-87745c55deb4-snapshots\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.737576 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.737554 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.737662 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.737578 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d940ecf2-11d4-4c5e-84de-87745c55deb4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.737946 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.737926 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/549e2d5f-64f0-438b-9d66-ccd547074e9d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:58.738090 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.738070 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bf1d832-d339-4c2e-a370-aa391743633c-trusted-ca\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.739033 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.739009 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d940ecf2-11d4-4c5e-84de-87745c55deb4-serving-cert\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.739113 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.739029 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf1d832-d339-4c2e-a370-aa391743633c-serving-cert\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.739113 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.739068 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c0aa75c-8b9a-46d8-86e9-492244a01b55-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.744767 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.744725 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qhmh\" (UniqueName: \"kubernetes.io/projected/5bf1d832-d339-4c2e-a370-aa391743633c-kube-api-access-4qhmh\") pod \"console-operator-68b757fb67-ztfgv\" (UID: \"5bf1d832-d339-4c2e-a370-aa391743633c\") " pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.745193 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.745172 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78fsw\" (UniqueName: \"kubernetes.io/projected/d940ecf2-11d4-4c5e-84de-87745c55deb4-kube-api-access-78fsw\") pod \"insights-operator-5c847dcfc4-hrmnd\" (UID: \"d940ecf2-11d4-4c5e-84de-87745c55deb4\") " pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.745457 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.745442 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9dd\" (UniqueName: \"kubernetes.io/projected/6c0aa75c-8b9a-46d8-86e9-492244a01b55-kube-api-access-gx9dd\") pod \"kube-storage-version-migrator-operator-846bf668f7-v7959\" (UID: \"6c0aa75c-8b9a-46d8-86e9-492244a01b55\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.750158 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.750103 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:09:58.800523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.800505 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:09:58.820043 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.820025 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:09:58.826313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.826292 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" Jul 08 21:09:58.832872 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.832853 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" Jul 08 21:09:58.837906 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.837884 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lgbp\" (UniqueName: \"kubernetes.io/projected/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-kube-api-access-7lgbp\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.838008 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.837960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfw2\" (UniqueName: \"kubernetes.io/projected/b429128e-1987-4465-a48c-219eaf47bd53-kube-api-access-hjfw2\") pod \"volume-data-source-validator-5d456fdbcd-j5fm2\" (UID: \"b429128e-1987-4465-a48c-219eaf47bd53\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" Jul 08 21:09:58.838072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.838019 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rtx\" (UniqueName: \"kubernetes.io/projected/c5656c5c-b830-41ca-9357-1ef150e63125-kube-api-access-96rtx\") pod \"network-check-source-989f8b8dc-s5zz4\" (UID: \"c5656c5c-b830-41ca-9357-1ef150e63125\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" Jul 08 21:09:58.838072 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.838044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.838317 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:58.838289 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 21:09:58.838395 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:58.838367 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls podName:e7dc49f0-f50c-4ed8-abb4-5838f79686fa nodeName:}" failed. No retries permitted until 2026-07-08 21:09:59.338346896 +0000 UTC m=+234.910091735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ptdvk" (UID: "e7dc49f0-f50c-4ed8-abb4-5838f79686fa") : secret "samples-operator-tls" not found Jul 08 21:09:58.846077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.846055 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rtx\" (UniqueName: \"kubernetes.io/projected/c5656c5c-b830-41ca-9357-1ef150e63125-kube-api-access-96rtx\") pod \"network-check-source-989f8b8dc-s5zz4\" (UID: \"c5656c5c-b830-41ca-9357-1ef150e63125\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" Jul 08 21:09:58.846523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.846505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjfw2\" (UniqueName: \"kubernetes.io/projected/b429128e-1987-4465-a48c-219eaf47bd53-kube-api-access-hjfw2\") pod \"volume-data-source-validator-5d456fdbcd-j5fm2\" (UID: \"b429128e-1987-4465-a48c-219eaf47bd53\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" Jul 08 21:09:58.847056 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.847039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lgbp\" (UniqueName: \"kubernetes.io/projected/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-kube-api-access-7lgbp\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:58.918780 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.918750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" Jul 08 21:09:58.936450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.936425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" Jul 08 21:09:58.962379 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.962294 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ztfgv"] Jul 08 21:09:58.966152 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:09:58.966119 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf1d832_d339_4c2e_a370_aa391743633c.slice/crio-c33ace1999a481f8349b3a1e724c2db44a71349e5492c6643db0b0769c1e54d4 WatchSource:0}: Error finding container c33ace1999a481f8349b3a1e724c2db44a71349e5492c6643db0b0769c1e54d4: Status 404 returned error can't find the container with id c33ace1999a481f8349b3a1e724c2db44a71349e5492c6643db0b0769c1e54d4 Jul 08 21:09:58.981727 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:58.981685 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959"] Jul 08 21:09:58.985952 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:09:58.985882 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0aa75c_8b9a_46d8_86e9_492244a01b55.slice/crio-bbbb301ddf21b6feb83d342f917f11517835a114cd9afecf25777cc39054859e WatchSource:0}: Error finding container bbbb301ddf21b6feb83d342f917f11517835a114cd9afecf25777cc39054859e: Status 404 returned error can't find the container with id bbbb301ddf21b6feb83d342f917f11517835a114cd9afecf25777cc39054859e Jul 08 21:09:59.055091 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.055058 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4"] Jul 08 21:09:59.058164 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:09:59.058129 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5656c5c_b830_41ca_9357_1ef150e63125.slice/crio-8c20b24f22c50ebac92a6abee3cf5288e454de9782701a473520a47e593a7a2b WatchSource:0}: Error finding container 8c20b24f22c50ebac92a6abee3cf5288e454de9782701a473520a47e593a7a2b: Status 404 returned error can't find the container with id 8c20b24f22c50ebac92a6abee3cf5288e454de9782701a473520a47e593a7a2b Jul 08 21:09:59.070187 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.070165 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2"] Jul 08 21:09:59.072234 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:09:59.072212 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb429128e_1987_4465_a48c_219eaf47bd53.slice/crio-97f2841c0af9298d53b9d2aa0cbd046311b77dbea0a2c09134694f9efab6d48e WatchSource:0}: Error finding container 97f2841c0af9298d53b9d2aa0cbd046311b77dbea0a2c09134694f9efab6d48e: Status 404 returned error can't find the container with id 97f2841c0af9298d53b9d2aa0cbd046311b77dbea0a2c09134694f9efab6d48e Jul 08 21:09:59.180626 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.180605 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hrmnd"] Jul 08 21:09:59.183074 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:09:59.183050 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd940ecf2_11d4_4c5e_84de_87745c55deb4.slice/crio-46cf151dd2a32021ecd0bf894e1f882d722986e7f75bcb159093032553784226 WatchSource:0}: Error finding container 46cf151dd2a32021ecd0bf894e1f882d722986e7f75bcb159093032553784226: Status 404 returned error can't find the container with id 46cf151dd2a32021ecd0bf894e1f882d722986e7f75bcb159093032553784226 Jul 08 21:09:59.240929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.240904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:09:59.241048 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:59.241033 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:09:59.241094 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:59.241088 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:00.241070565 +0000 UTC m=+235.812815423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:09:59.341594 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.341563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:09:59.341708 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:59.341691 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 21:09:59.341754 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:09:59.341745 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls podName:e7dc49f0-f50c-4ed8-abb4-5838f79686fa nodeName:}" failed. No retries permitted until 2026-07-08 21:10:00.341729503 +0000 UTC m=+235.913474360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ptdvk" (UID: "e7dc49f0-f50c-4ed8-abb4-5838f79686fa") : secret "samples-operator-tls" not found Jul 08 21:09:59.621754 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.621676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" event={"ID":"b429128e-1987-4465-a48c-219eaf47bd53","Type":"ContainerStarted","Data":"97f2841c0af9298d53b9d2aa0cbd046311b77dbea0a2c09134694f9efab6d48e"} Jul 08 21:09:59.623520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.623490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" event={"ID":"c5656c5c-b830-41ca-9357-1ef150e63125","Type":"ContainerStarted","Data":"a7514e5e0072f508afcb28f65128ba91de02a1f4b8ea1523fad41abf2fd5a329"} Jul 08 21:09:59.623647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.623538 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" event={"ID":"c5656c5c-b830-41ca-9357-1ef150e63125","Type":"ContainerStarted","Data":"8c20b24f22c50ebac92a6abee3cf5288e454de9782701a473520a47e593a7a2b"} Jul 08 21:09:59.624692 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.624664 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" event={"ID":"6c0aa75c-8b9a-46d8-86e9-492244a01b55","Type":"ContainerStarted","Data":"bbbb301ddf21b6feb83d342f917f11517835a114cd9afecf25777cc39054859e"} Jul 08 21:09:59.625917 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.625891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" event={"ID":"d940ecf2-11d4-4c5e-84de-87745c55deb4","Type":"ContainerStarted","Data":"46cf151dd2a32021ecd0bf894e1f882d722986e7f75bcb159093032553784226"} Jul 08 21:09:59.627515 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.627491 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" event={"ID":"5bf1d832-d339-4c2e-a370-aa391743633c","Type":"ContainerStarted","Data":"c33ace1999a481f8349b3a1e724c2db44a71349e5492c6643db0b0769c1e54d4"} Jul 08 21:09:59.638793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:09:59.638741 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-s5zz4" podStartSLOduration=1.638725102 podStartE2EDuration="1.638725102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:09:59.63780265 +0000 UTC m=+235.209547516" watchObservedRunningTime="2026-07-08 21:09:59.638725102 +0000 UTC m=+235.210469962" Jul 08 21:10:00.247600 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:00.247561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:00.247773 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:00.247721 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:10:00.247845 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:00.247792 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:02.247772017 +0000 UTC m=+237.819516854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:10:00.348583 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:00.348547 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:00.348755 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:00.348711 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 21:10:00.348825 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:00.348789 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls podName:e7dc49f0-f50c-4ed8-abb4-5838f79686fa nodeName:}" failed. No retries permitted until 2026-07-08 21:10:02.348767663 +0000 UTC m=+237.920512500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ptdvk" (UID: "e7dc49f0-f50c-4ed8-abb4-5838f79686fa") : secret "samples-operator-tls" not found Jul 08 21:10:01.635671 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:01.635632 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" event={"ID":"b429128e-1987-4465-a48c-219eaf47bd53","Type":"ContainerStarted","Data":"4c324cee3f928d57d0e05cca82172fcbbb6aa6869267b3e0ba4a5a181bed2ef9"} Jul 08 21:10:01.650626 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:01.650554 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j5fm2" podStartSLOduration=2.016740164 podStartE2EDuration="3.650534404s" totalImagesPullingTime="1.63379424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:01.650464341 +0000 UTC m=+237.222209200" watchObservedRunningTime="2026-07-08 21:10:01.650534404 +0000 UTC m=+237.222279265" Jul 08 21:10:01.805578 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:01.805537 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:02.262688 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.262644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:02.262858 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:02.262772 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:10:02.262918 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:02.262857 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:06.262834059 +0000 UTC m=+241.834578905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:10:02.363106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.363082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:02.363234 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:02.363216 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 21:10:02.363300 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:02.363289 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls podName:e7dc49f0-f50c-4ed8-abb4-5838f79686fa nodeName:}" failed. No retries permitted until 2026-07-08 21:10:06.363273914 +0000 UTC m=+241.935018751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ptdvk" (UID: "e7dc49f0-f50c-4ed8-abb4-5838f79686fa") : secret "samples-operator-tls" not found Jul 08 21:10:02.640340 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.640309 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" event={"ID":"6c0aa75c-8b9a-46d8-86e9-492244a01b55","Type":"ContainerStarted","Data":"65fe0cd83a96e6c98ab742c565de8afa7c3a96b1964df589b4750664636b7dec"} Jul 08 21:10:02.641945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.641919 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" event={"ID":"d940ecf2-11d4-4c5e-84de-87745c55deb4","Type":"ContainerStarted","Data":"3da9a7be8d5a12c60ff4027c63b1fdda9cc7f6ca2af0301b3ce5cdf7714afb79"} Jul 08 21:10:02.643540 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.643520 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/0.log" Jul 08 21:10:02.643641 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.643561 2565 generic.go:356] "Generic (PLEG): container finished" podID="5bf1d832-d339-4c2e-a370-aa391743633c" containerID="53e8b6d23415770128b1d6e2321d7c33406d3843c1e740093eb7130c55f59d18" exitCode=255 Jul 08 21:10:02.643641 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.643635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" event={"ID":"5bf1d832-d339-4c2e-a370-aa391743633c","Type":"ContainerDied","Data":"53e8b6d23415770128b1d6e2321d7c33406d3843c1e740093eb7130c55f59d18"} Jul 08 21:10:02.643812 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.643796 2565 scope.go:117] "RemoveContainer" containerID="53e8b6d23415770128b1d6e2321d7c33406d3843c1e740093eb7130c55f59d18" Jul 08 21:10:02.655086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.655032 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" podStartSLOduration=1.262850835 podStartE2EDuration="4.655014976s" totalImagesPullingTime="3.392164141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:02.653814056 +0000 UTC m=+238.225558914" watchObservedRunningTime="2026-07-08 21:10:02.655014976 +0000 UTC m=+238.226759839" Jul 08 21:10:02.670464 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:02.670426 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" podStartSLOduration=1.47845781 podStartE2EDuration="4.670410932s" totalImagesPullingTime="3.191953122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:02.670086393 +0000 UTC m=+238.241831253" watchObservedRunningTime="2026-07-08 21:10:02.670410932 +0000 UTC m=+238.242155792" Jul 08 21:10:03.648410 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.648385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:10:03.648819 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.648808 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/0.log" Jul 08 21:10:03.648885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.648851 2565 generic.go:356] "Generic (PLEG): container finished" podID="5bf1d832-d339-4c2e-a370-aa391743633c" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" exitCode=255 Jul 08 21:10:03.648964 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.648937 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" event={"ID":"5bf1d832-d339-4c2e-a370-aa391743633c","Type":"ContainerDied","Data":"132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15"} Jul 08 21:10:03.649015 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.649000 2565 scope.go:117] "RemoveContainer" containerID="53e8b6d23415770128b1d6e2321d7c33406d3843c1e740093eb7130c55f59d18" Jul 08 21:10:03.649192 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:03.649172 2565 scope.go:117] "RemoveContainer" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" Jul 08 21:10:03.649389 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:03.649363 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ztfgv_openshift-console-operator(5bf1d832-d339-4c2e-a370-aa391743633c)\"" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" podUID="5bf1d832-d339-4c2e-a370-aa391743633c" Jul 08 21:10:04.002094 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.001999 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.021213 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.021186 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc"] Jul 08 21:10:04.023747 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.023729 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" Jul 08 21:10:04.026097 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.026076 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r4bj8\"" Jul 08 21:10:04.026442 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.026424 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 21:10:04.026517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.026455 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 21:10:04.033499 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.033481 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc"] Jul 08 21:10:04.083155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.083116 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.135790 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.135755 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.168835 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.168806 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r6lhx"] Jul 08 21:10:04.171155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.171138 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.173530 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.173507 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 21:10:04.173643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.173568 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 21:10:04.173643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.173568 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zz8jg\"" Jul 08 21:10:04.179819 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.179797 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk7mm\" (UniqueName: \"kubernetes.io/projected/cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8-kube-api-access-rk7mm\") pod \"migrator-d57d58cc9-dpdqc\" (UID: \"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" Jul 08 21:10:04.180767 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.180750 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r6lhx"] Jul 08 21:10:04.214404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.214383 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.281017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.280997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.281109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.281057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.281109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.281082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e99c6906-ce61-4faa-ba3f-160952a55a2d-crio-socket\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.281184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.281120 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7mm\" (UniqueName: \"kubernetes.io/projected/cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8-kube-api-access-rk7mm\") pod \"migrator-d57d58cc9-dpdqc\" (UID: \"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" Jul 08 21:10:04.281184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.281153 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e99c6906-ce61-4faa-ba3f-160952a55a2d-data-volume\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.281184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.281168 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmkk\" (UniqueName: \"kubernetes.io/projected/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-api-access-8nmkk\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.287851 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.287833 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk7mm\" (UniqueName: \"kubernetes.io/projected/cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8-kube-api-access-rk7mm\") pod \"migrator-d57d58cc9-dpdqc\" (UID: \"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" Jul 08 21:10:04.332878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.332859 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" Jul 08 21:10:04.381699 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381667 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e99c6906-ce61-4faa-ba3f-160952a55a2d-data-volume\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.381804 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmkk\" (UniqueName: \"kubernetes.io/projected/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-api-access-8nmkk\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.381804 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.381878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.381940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e99c6906-ce61-4faa-ba3f-160952a55a2d-crio-socket\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.382003 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.381988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e99c6906-ce61-4faa-ba3f-160952a55a2d-crio-socket\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.382280 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.382063 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:10:04.382280 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.382136 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:04.882116044 +0000 UTC m=+240.453860904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : configmap "kube-rbac-proxy" not found Jul 08 21:10:04.382280 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.382176 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:10:04.382280 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.382263 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:04.882226428 +0000 UTC m=+240.453971268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : secret "insights-runtime-extractor-tls" not found Jul 08 21:10:04.382623 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.382233 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e99c6906-ce61-4faa-ba3f-160952a55a2d-data-volume\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.390380 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.390347 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmkk\" (UniqueName: \"kubernetes.io/projected/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-api-access-8nmkk\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.450044 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.450020 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc"] Jul 08 21:10:04.452359 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:04.452334 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdaed41a_f9c3_43c1_8eff_6055a5c4dcc8.slice/crio-36eb10f8d366f36fbcc4e9b9c0250793a588d16a8d92c69f2b85ec4d6b791801 WatchSource:0}: Error finding container 36eb10f8d366f36fbcc4e9b9c0250793a588d16a8d92c69f2b85ec4d6b791801: Status 404 returned error can't find the container with id 36eb10f8d366f36fbcc4e9b9c0250793a588d16a8d92c69f2b85ec4d6b791801 Jul 08 21:10:04.643221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.643167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.652589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.652574 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:10:04.652930 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.652884 2565 scope.go:117] "RemoveContainer" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" Jul 08 21:10:04.653054 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.653031 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ztfgv_openshift-console-operator(5bf1d832-d339-4c2e-a370-aa391743633c)\"" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" podUID="5bf1d832-d339-4c2e-a370-aa391743633c" Jul 08 21:10:04.653724 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.653705 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" event={"ID":"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8","Type":"ContainerStarted","Data":"36eb10f8d366f36fbcc4e9b9c0250793a588d16a8d92c69f2b85ec4d6b791801"} Jul 08 21:10:04.885536 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.885507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.885645 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.885569 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:04.885645 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.885612 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:10:04.885736 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.885671 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:05.885652997 +0000 UTC m=+241.457397849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : configmap "kube-rbac-proxy" not found Jul 08 21:10:04.885736 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.885685 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:10:04.885736 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:04.885721 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:05.885709972 +0000 UTC m=+241.457454811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : secret "insights-runtime-extractor-tls" not found Jul 08 21:10:04.923463 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.923398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:04.973365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:04.973341 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:05.892165 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:05.892135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:05.892633 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:05.892197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:05.892633 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:05.892286 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:10:05.892633 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:05.892348 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:10:05.892633 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:05.892366 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:07.892345544 +0000 UTC m=+243.464090382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : configmap "kube-rbac-proxy" not found Jul 08 21:10:05.892633 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:05.892396 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:07.892384246 +0000 UTC m=+243.464129106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : secret "insights-runtime-extractor-tls" not found Jul 08 21:10:06.295904 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:06.295874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:06.296060 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:06.296017 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:10:06.296111 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:06.296095 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:14.296074694 +0000 UTC m=+249.867819533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:10:06.396845 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:06.396822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:06.396988 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:06.396946 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 21:10:06.397052 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:06.397005 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls podName:e7dc49f0-f50c-4ed8-abb4-5838f79686fa nodeName:}" failed. No retries permitted until 2026-07-08 21:10:14.39698841 +0000 UTC m=+249.968733247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ptdvk" (UID: "e7dc49f0-f50c-4ed8-abb4-5838f79686fa") : secret "samples-operator-tls" not found Jul 08 21:10:06.541829 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:06.541801 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:06.588249 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:06.588171 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:07.069177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.069151 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4ccx_a1c66415-3f68-4a2a-9137-f3c95eb1cd13/dns-node-resolver/0.log" Jul 08 21:10:07.371041 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.371014 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:07.663259 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.663183 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" event={"ID":"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8","Type":"ContainerStarted","Data":"5a41ccdae312109a522fbc8eaddfd73151cbd05e90279a5255046e6a93f5c1c1"} Jul 08 21:10:07.663259 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.663252 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" event={"ID":"cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8","Type":"ContainerStarted","Data":"638ff1ab91ef0c3891c9d891d91466a71d5ed00287fc2309ba0c5367b361910a"} Jul 08 21:10:07.682351 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.680013 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dpdqc" podStartSLOduration=1.274531411 podStartE2EDuration="3.679994239s" totalImagesPullingTime="2.405462828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:07.67814048 +0000 UTC m=+243.249885339" watchObservedRunningTime="2026-07-08 21:10:07.679994239 +0000 UTC m=+243.251739099" Jul 08 21:10:07.869130 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.869107 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl9b5_56d4bc64-7000-4b7a-bbd5-ad884d2e7262/node-ca/0.log" Jul 08 21:10:07.909512 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.909487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:07.909585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:07.909540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:07.909618 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:07.909602 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:10:07.909661 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:07.909623 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:10:07.909691 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:07.909661 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:11.909646034 +0000 UTC m=+247.481390889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : configmap "kube-rbac-proxy" not found Jul 08 21:10:07.909691 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:07.909676 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:11.909669423 +0000 UTC m=+247.481414260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : secret "insights-runtime-extractor-tls" not found Jul 08 21:10:08.395957 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.395930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:08.498583 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.498556 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:08.597637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.597616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:08.647479 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.647419 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:08.723738 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.723682 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:08.821041 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.821021 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:10:08.821132 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.821115 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:10:08.821387 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.821375 2565 scope.go:117] "RemoveContainer" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" Jul 08 21:10:08.821512 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:08.821495 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ztfgv_openshift-console-operator(5bf1d832-d339-4c2e-a370-aa391743633c)\"" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" podUID="5bf1d832-d339-4c2e-a370-aa391743633c" Jul 08 21:10:08.869502 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:08.869478 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p578w_b787178f-18f3-4d58-888d-760a65129311/serve-healthcheck-canary/0.log" Jul 08 21:10:09.068989 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.068965 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dpdqc_cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8/migrator/0.log" Jul 08 21:10:09.099075 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.099049 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:09.268003 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.267983 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dpdqc_cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8/graceful-termination/0.log" Jul 08 21:10:09.288410 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.288385 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:09.470542 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.470482 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-v7959_6c0aa75c-8b9a-46d8-86e9-492244a01b55/kube-storage-version-migrator-operator/0.log" Jul 08 21:10:09.669531 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.669513 2565 scope.go:117] "RemoveContainer" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" Jul 08 21:10:09.669656 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:09.669639 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ztfgv_openshift-console-operator(5bf1d832-d339-4c2e-a370-aa391743633c)\"" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" podUID="5bf1d832-d339-4c2e-a370-aa391743633c" Jul 08 21:10:09.889847 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:09.889825 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:10.393999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:10.393973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:10.497369 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:10.497343 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:10.688276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:10.688202 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:10.891121 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:10.891097 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:11.292567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:11.292532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:11.901700 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:11.901677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:11.939216 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:11.939190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:11.939343 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:11.939282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:11.939343 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:11.939300 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:10:11.939450 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:11.939347 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:19.939334291 +0000 UTC m=+255.511079127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : configmap "kube-rbac-proxy" not found Jul 08 21:10:11.939450 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:11.939396 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:10:11.939535 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:11.939454 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls podName:e99c6906-ce61-4faa-ba3f-160952a55a2d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:19.93943706 +0000 UTC m=+255.511181911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r6lhx" (UID: "e99c6906-ce61-4faa-ba3f-160952a55a2d") : secret "insights-runtime-extractor-tls" not found Jul 08 21:10:12.286839 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:12.286818 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:12.419558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:12.419528 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:12.520578 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:12.520557 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:12.692602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:12.692534 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:12.889255 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:12.889212 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:13.288647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:13.288619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:14.149548 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.149498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:14.288297 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.288266 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:14.356546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.356522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:14.356848 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:14.356656 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:10:14.356848 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:14.356707 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert podName:549e2d5f-64f0-438b-9d66-ccd547074e9d nodeName:}" failed. No retries permitted until 2026-07-08 21:10:30.356694384 +0000 UTC m=+265.928439221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-898wv" (UID: "549e2d5f-64f0-438b-9d66-ccd547074e9d") : secret "networking-console-plugin-cert" not found Jul 08 21:10:14.457040 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.456989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:14.459305 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.459288 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7dc49f0-f50c-4ed8-abb4-5838f79686fa-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ptdvk\" (UID: \"e7dc49f0-f50c-4ed8-abb4-5838f79686fa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:14.533411 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.533389 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cpzst\"" Jul 08 21:10:14.542282 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.542265 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" Jul 08 21:10:14.662029 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.662003 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk"] Jul 08 21:10:14.690430 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.690404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:14.892299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.892276 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:14.939699 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:14.939675 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:15.490359 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.490331 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:15.686667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.686639 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" event={"ID":"e7dc49f0-f50c-4ed8-abb4-5838f79686fa","Type":"ContainerStarted","Data":"8a64bf9257335fe348012eec56ebf311e56ba4e3694b1e51e812285a5498cce2"} Jul 08 21:10:15.767067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.766999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:10:15.769473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.769446 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3-metrics-certs\") pod \"network-metrics-daemon-rz5lw\" (UID: \"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3\") " pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:10:15.847723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.847700 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfx9c\"" Jul 08 21:10:15.855825 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.855800 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz5lw" Jul 08 21:10:15.888877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.888832 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:15.974428 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:15.974404 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rz5lw"] Jul 08 21:10:15.977394 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:15.977355 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2c37d4_5fbd_4a6c_8e6e_6d3aa631c2c3.slice/crio-c3f953b3719f6bcb261a206f3555555962c2c83826ec73263776a145a91eb958 WatchSource:0}: Error finding container c3f953b3719f6bcb261a206f3555555962c2c83826ec73263776a145a91eb958: Status 404 returned error can't find the container with id c3f953b3719f6bcb261a206f3555555962c2c83826ec73263776a145a91eb958 Jul 08 21:10:16.296329 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:16.296292 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:16.485388 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:16.485350 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:16.692206 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:16.692127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz5lw" event={"ID":"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3","Type":"ContainerStarted","Data":"c3f953b3719f6bcb261a206f3555555962c2c83826ec73263776a145a91eb958"} Jul 08 21:10:16.891405 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:16.891371 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:17.275558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.275528 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:17.488585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.488557 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:17.696767 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.696735 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" event={"ID":"e7dc49f0-f50c-4ed8-abb4-5838f79686fa","Type":"ContainerStarted","Data":"1b4a8d6d326885417f4560ee4879fe922edb6c30bdfb1ade8a8ddcc5908c2628"} Jul 08 21:10:17.697157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.696784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" event={"ID":"e7dc49f0-f50c-4ed8-abb4-5838f79686fa","Type":"ContainerStarted","Data":"ebf7105bf52feddfd79841ff42a3a104e487d0eb80a75ccb0207cf15e1c20911"} Jul 08 21:10:17.698394 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.698373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz5lw" event={"ID":"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3","Type":"ContainerStarted","Data":"e915f10e76f834dd0cfd872d8e19241f24d8d6a854096a2a3913f6c85a14d236"} Jul 08 21:10:17.698471 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.698417 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz5lw" event={"ID":"fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3","Type":"ContainerStarted","Data":"0df374d92002e18db3cc9f8bab6b369f7cd5469a07dab1ce904f3cb2909917cd"} Jul 08 21:10:17.712067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.712022 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ptdvk" podStartSLOduration=17.105172678 podStartE2EDuration="19.712010204s" totalImagesPullingTime="2.606837526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:17.711549797 +0000 UTC m=+253.283294668" watchObservedRunningTime="2026-07-08 21:10:17.712010204 +0000 UTC m=+253.283755063" Jul 08 21:10:17.725360 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.725322 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rz5lw" podStartSLOduration=251.393073493 podStartE2EDuration="4m12.725311316s" totalImagesPullingTime="1.332237823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:17.724512073 +0000 UTC m=+253.296256933" watchObservedRunningTime="2026-07-08 21:10:17.725311316 +0000 UTC m=+253.297056175" Jul 08 21:10:17.890344 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:17.890316 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:18.287290 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:18.287233 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:18.494741 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:18.494714 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:18.688080 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:18.688003 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:18.739884 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:18.739859 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:19.091402 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:19.091376 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:20.002498 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.002467 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:20.002935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.002528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:20.002935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.002918 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e99c6906-ce61-4faa-ba3f-160952a55a2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:20.004832 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.004801 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e99c6906-ce61-4faa-ba3f-160952a55a2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r6lhx\" (UID: \"e99c6906-ce61-4faa-ba3f-160952a55a2d\") " pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:20.081489 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.081468 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zz8jg\"" Jul 08 21:10:20.089971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.089952 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r6lhx" Jul 08 21:10:20.237953 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.237927 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r6lhx"] Jul 08 21:10:20.240809 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:20.240785 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99c6906_ce61_4faa_ba3f_160952a55a2d.slice/crio-c5374760327f7c118a440e4fd7f9e98871833db2e01a25c97cf8e5042c580fae WatchSource:0}: Error finding container c5374760327f7c118a440e4fd7f9e98871833db2e01a25c97cf8e5042c580fae: Status 404 returned error can't find the container with id c5374760327f7c118a440e4fd7f9e98871833db2e01a25c97cf8e5042c580fae Jul 08 21:10:20.709201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:20.709160 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r6lhx" event={"ID":"e99c6906-ce61-4faa-ba3f-160952a55a2d","Type":"ContainerStarted","Data":"c5374760327f7c118a440e4fd7f9e98871833db2e01a25c97cf8e5042c580fae"} Jul 08 21:10:21.944249 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:21.944206 2565 scope.go:117] "RemoveContainer" containerID="132fe843b818effa237cdee2fa89423d1343e84f2f633ef58a27a30223d98c15" Jul 08 21:10:22.561563 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.561527 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:22.671726 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.671691 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:22.718819 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.718797 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:10:22.718971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.718858 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" event={"ID":"5bf1d832-d339-4c2e-a370-aa391743633c","Type":"ContainerStarted","Data":"52bef7b38d8c4126f7a8c1cd281339ccc3f9e30c9fc1e5d57cd490a4c7232054"} Jul 08 21:10:22.719163 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.719140 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:10:22.724028 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.724005 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" Jul 08 21:10:22.734672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.734624 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-ztfgv" podStartSLOduration=21.332277476 podStartE2EDuration="24.734606477s" totalImagesPullingTime="3.402329001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:22.73360154 +0000 UTC m=+258.305346398" watchObservedRunningTime="2026-07-08 21:10:22.734606477 +0000 UTC m=+258.306351336" Jul 08 21:10:22.879384 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:22.879260 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:23.047124 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:23.047088 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:23.105893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:23.105856 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:23.451154 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:23.451114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:23.659683 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:23.659642 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:23.843401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:23.843368 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.059046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.059007 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.203260 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.203163 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.303770 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.303733 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.351943 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.351912 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.502811 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.502782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.727180 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.727153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r6lhx" event={"ID":"e99c6906-ce61-4faa-ba3f-160952a55a2d","Type":"ContainerStarted","Data":"5a5f7879136237030db1df3f07930bd7cfb91495abaae445fd0f1ff1dc67c457"} Jul 08 21:10:24.885617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.885581 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:24.940372 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:24.940188 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:25.148800 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.148735 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-ql4h7"] Jul 08 21:10:25.150894 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.150875 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:25.153224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.153207 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 21:10:25.153363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.153207 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 21:10:25.153363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.153267 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tqxzm\"" Jul 08 21:10:25.161640 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.161620 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ql4h7"] Jul 08 21:10:25.211481 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.211448 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:25.246638 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.246616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpv4j\" (UniqueName: \"kubernetes.io/projected/4ec95395-3efe-40f4-87bd-72afc81c739b-kube-api-access-zpv4j\") pod \"downloads-685bfb5f64-ql4h7\" (UID: \"4ec95395-3efe-40f4-87bd-72afc81c739b\") " pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:25.311742 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.311714 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:25.346982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.346959 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpv4j\" (UniqueName: \"kubernetes.io/projected/4ec95395-3efe-40f4-87bd-72afc81c739b-kube-api-access-zpv4j\") pod \"downloads-685bfb5f64-ql4h7\" (UID: \"4ec95395-3efe-40f4-87bd-72afc81c739b\") " pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:25.356579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.356555 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpv4j\" (UniqueName: \"kubernetes.io/projected/4ec95395-3efe-40f4-87bd-72afc81c739b-kube-api-access-zpv4j\") pod \"downloads-685bfb5f64-ql4h7\" (UID: \"4ec95395-3efe-40f4-87bd-72afc81c739b\") " pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:25.413931 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.413861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:25.459573 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.459547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:25.601286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.601262 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ql4h7"] Jul 08 21:10:25.602110 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:25.602081 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec95395_3efe_40f4_87bd_72afc81c739b.slice/crio-47fd59c7ad1504ac64f6d5a24145f9a34aa34caa2981061cb7c51604e7e0f72c WatchSource:0}: Error finding container 47fd59c7ad1504ac64f6d5a24145f9a34aa34caa2981061cb7c51604e7e0f72c: Status 404 returned error can't find the container with id 47fd59c7ad1504ac64f6d5a24145f9a34aa34caa2981061cb7c51604e7e0f72c Jul 08 21:10:25.731982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.731907 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ql4h7" event={"ID":"4ec95395-3efe-40f4-87bd-72afc81c739b","Type":"ContainerStarted","Data":"47fd59c7ad1504ac64f6d5a24145f9a34aa34caa2981061cb7c51604e7e0f72c"} Jul 08 21:10:25.733850 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:25.733822 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r6lhx" event={"ID":"e99c6906-ce61-4faa-ba3f-160952a55a2d","Type":"ContainerStarted","Data":"fb1c41c569994cdc66b3081dac75e2a050abf10f8dd3cf5f953919293fa8af58"} Jul 08 21:10:27.743193 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:27.743153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r6lhx" event={"ID":"e99c6906-ce61-4faa-ba3f-160952a55a2d","Type":"ContainerStarted","Data":"6318e82e6bac25a5fa2994b5bf67fa8ac522dbe12c16456c649d9d33de8f1d1d"} Jul 08 21:10:27.761084 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:27.760975 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r6lhx" podStartSLOduration=16.59610019 podStartE2EDuration="23.76095482s" totalImagesPullingTime="7.16485463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:10:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:27.759046179 +0000 UTC m=+263.330791040" watchObservedRunningTime="2026-07-08 21:10:27.76095482 +0000 UTC m=+263.332699680" Jul 08 21:10:30.384709 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.384667 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:30.386869 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.386849 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/549e2d5f-64f0-438b-9d66-ccd547074e9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-898wv\" (UID: \"549e2d5f-64f0-438b-9d66-ccd547074e9d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:30.614018 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.613991 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qg55n\"" Jul 08 21:10:30.621823 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.621803 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" Jul 08 21:10:30.746127 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.746002 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-898wv"] Jul 08 21:10:30.748295 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:30.748262 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod549e2d5f_64f0_438b_9d66_ccd547074e9d.slice/crio-93a4dceb854c8bb7f60aefbcd6afe85bd161fae9b7f1deef03383dbd0d70d7ab WatchSource:0}: Error finding container 93a4dceb854c8bb7f60aefbcd6afe85bd161fae9b7f1deef03383dbd0d70d7ab: Status 404 returned error can't find the container with id 93a4dceb854c8bb7f60aefbcd6afe85bd161fae9b7f1deef03383dbd0d70d7ab Jul 08 21:10:30.754741 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:30.754717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" event={"ID":"549e2d5f-64f0-438b-9d66-ccd547074e9d","Type":"ContainerStarted","Data":"93a4dceb854c8bb7f60aefbcd6afe85bd161fae9b7f1deef03383dbd0d70d7ab"} Jul 08 21:10:31.282642 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:31.282598 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:31.336668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:31.336627 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:32.762988 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:32.762950 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" event={"ID":"549e2d5f-64f0-438b-9d66-ccd547074e9d","Type":"ContainerStarted","Data":"9ddd0721b63dd85f441d5bd496eaa31dc6dca481e609e9d4fc174c32afb87075"} Jul 08 21:10:32.777047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:32.777008 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-898wv" podStartSLOduration=33.483443401 podStartE2EDuration="34.776995525s" totalImagesPullingTime="1.293552124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:32.776601516 +0000 UTC m=+268.348346375" watchObservedRunningTime="2026-07-08 21:10:32.776995525 +0000 UTC m=+268.348740384" Jul 08 21:10:32.859286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:32.859251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:32.913804 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:32.913773 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:33.332512 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.332396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:33.433138 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.433106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:33.484469 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.484445 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9"] Jul 08 21:10:33.487728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.487712 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:33.489824 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.489806 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 21:10:33.489925 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.489899 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wsrgm\"" Jul 08 21:10:33.493590 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.493570 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9"] Jul 08 21:10:33.497004 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.496979 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:33.609687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.609624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28e67d46-c60c-4d74-902f-e755e059f45b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x5sr9\" (UID: \"28e67d46-c60c-4d74-902f-e755e059f45b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:33.617954 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.617930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:33.710144 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.710120 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28e67d46-c60c-4d74-902f-e755e059f45b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x5sr9\" (UID: \"28e67d46-c60c-4d74-902f-e755e059f45b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:33.712430 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.712407 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28e67d46-c60c-4d74-902f-e755e059f45b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x5sr9\" (UID: \"28e67d46-c60c-4d74-902f-e755e059f45b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:33.796843 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.796819 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:33.933965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:33.933950 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9"] Jul 08 21:10:33.936124 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:33.936100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e67d46_c60c_4d74_902f_e755e059f45b.slice/crio-9e9cab42ecb4e79acaf3c34fd3f256ebcf371ba2ad828a211bcd7531dc469f10 WatchSource:0}: Error finding container 9e9cab42ecb4e79acaf3c34fd3f256ebcf371ba2ad828a211bcd7531dc469f10: Status 404 returned error can't find the container with id 9e9cab42ecb4e79acaf3c34fd3f256ebcf371ba2ad828a211bcd7531dc469f10 Jul 08 21:10:34.771664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:34.771623 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" event={"ID":"28e67d46-c60c-4d74-902f-e755e059f45b","Type":"ContainerStarted","Data":"9e9cab42ecb4e79acaf3c34fd3f256ebcf371ba2ad828a211bcd7531dc469f10"} Jul 08 21:10:35.776302 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:35.776261 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" event={"ID":"28e67d46-c60c-4d74-902f-e755e059f45b","Type":"ContainerStarted","Data":"bc77376ecb5566b9a23206faa22befd0cbedfc913330cc698162e71de944a403"} Jul 08 21:10:35.776729 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:35.776489 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:35.782121 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:35.782096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" Jul 08 21:10:35.790720 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:35.790678 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x5sr9" podStartSLOduration=1.469640609 podStartE2EDuration="2.790661332s" totalImagesPullingTime="1.321020723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:35.790208988 +0000 UTC m=+271.361953851" watchObservedRunningTime="2026-07-08 21:10:35.790661332 +0000 UTC m=+271.362406170" Jul 08 21:10:36.524961 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.524914 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:36.542443 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.542415 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5"] Jul 08 21:10:36.545885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.545867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.548319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.548250 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 21:10:36.548319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.548305 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 21:10:36.549560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.549540 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t6h2x\"" Jul 08 21:10:36.549667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.549559 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 21:10:36.549729 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.549714 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 21:10:36.549814 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.549792 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 21:10:36.554177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.554155 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5"] Jul 08 21:10:36.592650 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.592620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:36.631382 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.631358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.631495 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.631395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsxc5\" (UniqueName: \"kubernetes.io/projected/f6210c44-7749-42e2-9184-bb1d279b2a6e-kube-api-access-vsxc5\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.631495 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.631439 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6210c44-7749-42e2-9184-bb1d279b2a6e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.631591 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.631530 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.713428 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.713396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:36.731853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.731831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.731979 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.731889 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.731979 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.731921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsxc5\" (UniqueName: \"kubernetes.io/projected/f6210c44-7749-42e2-9184-bb1d279b2a6e-kube-api-access-vsxc5\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.732102 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.732079 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6210c44-7749-42e2-9184-bb1d279b2a6e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.732853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.732829 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6210c44-7749-42e2-9184-bb1d279b2a6e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.734598 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.734580 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.734771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.734745 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6210c44-7749-42e2-9184-bb1d279b2a6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.739407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.739386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsxc5\" (UniqueName: \"kubernetes.io/projected/f6210c44-7749-42e2-9184-bb1d279b2a6e-kube-api-access-vsxc5\") pod \"prometheus-operator-567f7b8b7c-f4gb5\" (UID: \"f6210c44-7749-42e2-9184-bb1d279b2a6e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.858416 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.858391 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" Jul 08 21:10:36.994354 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:36.994322 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5"] Jul 08 21:10:36.997230 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:36.997199 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6210c44_7749_42e2_9184_bb1d279b2a6e.slice/crio-0d1a2227c7a7fb1c60d155287825d8b867a55270679e470c177a23822122cf7f WatchSource:0}: Error finding container 0d1a2227c7a7fb1c60d155287825d8b867a55270679e470c177a23822122cf7f: Status 404 returned error can't find the container with id 0d1a2227c7a7fb1c60d155287825d8b867a55270679e470c177a23822122cf7f Jul 08 21:10:37.474759 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:37.474723 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:37.786045 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:37.785973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" event={"ID":"f6210c44-7749-42e2-9184-bb1d279b2a6e","Type":"ContainerStarted","Data":"0d1a2227c7a7fb1c60d155287825d8b867a55270679e470c177a23822122cf7f"} Jul 08 21:10:38.793348 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:38.793308 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" event={"ID":"f6210c44-7749-42e2-9184-bb1d279b2a6e","Type":"ContainerStarted","Data":"98fbd707dd0ce717997eaa32e92c665ef1e96346aea1a5985f09fb723c1182a1"} Jul 08 21:10:38.793791 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:38.793377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" event={"ID":"f6210c44-7749-42e2-9184-bb1d279b2a6e","Type":"ContainerStarted","Data":"c11f1e1af9d930dfae41a76801e872ee6ab3efa36a94e70a1e8fa1d07da24230"} Jul 08 21:10:38.811074 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:38.810956 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-f4gb5" podStartSLOduration=1.2533846149999999 podStartE2EDuration="2.810936644s" totalImagesPullingTime="1.557552029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:38.809996938 +0000 UTC m=+274.381741798" watchObservedRunningTime="2026-07-08 21:10:38.810936644 +0000 UTC m=+274.382681505" Jul 08 21:10:38.842768 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:38.842718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:39.277866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:39.277785 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:39.326364 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:39.326328 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:40.631716 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.631681 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:40.727549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.727512 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:40.854858 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.854824 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:40.896102 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.896033 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-22nr7"] Jul 08 21:10:40.900949 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.900928 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.903208 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.903132 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 21:10:40.903208 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.903146 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 21:10:40.903415 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.903310 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vmxqb\"" Jul 08 21:10:40.903498 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.903441 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 21:10:40.939035 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.939002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:40.969354 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969334 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969465 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-root\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969465 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-wtmp\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-textfile\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969548 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-metrics-client-ca\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-sys\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969727 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-tls\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.969727 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.969697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8fcs\" (UniqueName: \"kubernetes.io/projected/113e034e-b9c8-4d67-807f-4e32c414fa3b-kube-api-access-g8fcs\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:40.999182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:40.999138 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.070233 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-root\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070233 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070252 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-wtmp\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-textfile\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070313 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-root\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-metrics-client-ca\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-sys\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070455 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-wtmp\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070497 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e034e-b9c8-4d67-807f-4e32c414fa3b-sys\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070512 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-tls\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8fcs\" (UniqueName: \"kubernetes.io/projected/113e034e-b9c8-4d67-807f-4e32c414fa3b-kube-api-access-g8fcs\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-textfile\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.070963 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.070938 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-metrics-client-ca\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.071035 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.071011 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.073206 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.073168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-tls\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.073206 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.073176 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/113e034e-b9c8-4d67-807f-4e32c414fa3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.077980 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.077959 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8fcs\" (UniqueName: \"kubernetes.io/projected/113e034e-b9c8-4d67-807f-4e32c414fa3b-kube-api-access-g8fcs\") pod \"node-exporter-22nr7\" (UID: \"113e034e-b9c8-4d67-807f-4e32c414fa3b\") " pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.096060 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.096033 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.198994 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.198840 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.211369 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.211345 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22nr7" Jul 08 21:10:41.297930 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.297898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.398274 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.398211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.510077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.509997 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.855099 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.855055 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.882925 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.882893 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:10:41.886211 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.886190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.888615 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.888596 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 21:10:41.888725 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.888657 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r54dz\"" Jul 08 21:10:41.889417 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.889398 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 21:10:41.889512 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.889461 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 21:10:41.889886 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.889850 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 21:10:41.890096 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.890071 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 21:10:41.895374 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.895349 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 21:10:41.896230 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.896212 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:10:41.943140 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.943075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:41.949918 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.949889 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:10:41.953941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.953913 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:41.956721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.956574 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 21:10:41.956721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.956598 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 21:10:41.956721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.956598 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 21:10:41.956721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.956601 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 21:10:41.956721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.956690 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 21:10:41.957015 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.957000 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k6htb\"" Jul 08 21:10:41.957186 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.957168 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 21:10:41.957186 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.957192 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 21:10:41.957456 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.957428 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 21:10:41.958187 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.958029 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 21:10:41.966054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.965819 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:10:41.978936 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.978911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.978960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.979049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.979131 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.979157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kgx\" (UniqueName: \"kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979396 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.979226 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:41.979396 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:41.979261 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080130 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080348 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080380 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8p8t\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080512 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080574 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.080956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080656 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kgx\" (UniqueName: \"kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.080956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.080893 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.081253 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.081211 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.081358 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.081330 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.081432 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.081383 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.081432 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.081385 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.083361 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.083341 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.083467 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.083436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.089162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.089139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kgx\" (UniqueName: \"kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx\") pod \"console-76989678c5-mb7tz\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.156503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.156433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:42.181525 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181525 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181696 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8p8t\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181696 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181696 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181659 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181696 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181681 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.181991 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.181909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182066 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182137 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182137 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182349 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:42.182049 2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 21:10:42.182404 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:42.182392 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls podName:a3676d57-11c7-4391-b192-d374a9464e02 nodeName:}" failed. No retries permitted until 2026-07-08 21:10:42.682370787 +0000 UTC m=+278.254115624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a3676d57-11c7-4391-b192-d374a9464e02") : secret "alertmanager-main-tls" not found Jul 08 21:10:42.182480 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182441 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.182539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.182514 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.183607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.183376 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.185450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185416 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.185450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185432 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185608 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185608 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.185912 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186840 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.186817 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.186993 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.186965 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.190392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.190369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8p8t\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.198134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.198093 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:10:42.255872 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.255836 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:42.316687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.316622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:42.445146 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:42.445056 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" Jul 08 21:10:42.445146 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:10:42.445056 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5926b" podUID="029c9231-733f-43bd-b92c-17b495c2adeb" Jul 08 21:10:42.686418 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.686377 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.689007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.688981 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:42.810635 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.810606 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:10:42.810635 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.810643 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5926b" Jul 08 21:10:42.869420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:42.869377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:10:43.687107 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:43.687072 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:43.786930 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:43.786898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:43.834080 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:43.834044 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:43.939647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:43.939558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:43.990763 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:43.990731 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:44.057901 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:44.057862 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:45.269074 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.269041 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-74f69594c9-k2jzc"] Jul 08 21:10:45.272520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.272496 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.274951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.274925 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 21:10:45.275086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.275037 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 21:10:45.275142 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.275103 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 21:10:45.276215 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.276131 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 21:10:45.276215 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.276130 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pm4fc\"" Jul 08 21:10:45.276215 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.276212 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-36ps2gpg97q5t\"" Jul 08 21:10:45.282941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.282894 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74f69594c9-k2jzc"] Jul 08 21:10:45.412533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-client-certs\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412698 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412698 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c0149484-2084-4ab5-9ff4-e7f737e32494-audit-log\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412698 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx54h\" (UniqueName: \"kubernetes.io/projected/c0149484-2084-4ab5-9ff4-e7f737e32494-kube-api-access-hx54h\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412698 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-tls\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412707 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-client-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.412900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.412731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-metrics-server-audit-profiles\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514264 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c0149484-2084-4ab5-9ff4-e7f737e32494-audit-log\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514264 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514269 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx54h\" (UniqueName: \"kubernetes.io/projected/c0149484-2084-4ab5-9ff4-e7f737e32494-kube-api-access-hx54h\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-tls\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-client-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-metrics-server-audit-profiles\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-client-certs\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514612 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.514742 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.514661 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c0149484-2084-4ab5-9ff4-e7f737e32494-audit-log\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.515344 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.515318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.515625 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.515597 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c0149484-2084-4ab5-9ff4-e7f737e32494-metrics-server-audit-profiles\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.517340 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.517319 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-tls\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.517524 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.517503 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-secret-metrics-server-client-certs\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.517925 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.517904 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0149484-2084-4ab5-9ff4-e7f737e32494-client-ca-bundle\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.521457 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.521403 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx54h\" (UniqueName: \"kubernetes.io/projected/c0149484-2084-4ab5-9ff4-e7f737e32494-kube-api-access-hx54h\") pod \"metrics-server-74f69594c9-k2jzc\" (UID: \"c0149484-2084-4ab5-9ff4-e7f737e32494\") " pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.586263 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.586223 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:10:45.692990 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.692941 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:45.716945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.716911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:10:45.717087 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.716981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:10:45.719750 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.719720 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029c9231-733f-43bd-b92c-17b495c2adeb-metrics-tls\") pod \"dns-default-5926b\" (UID: \"029c9231-733f-43bd-b92c-17b495c2adeb\") " pod="openshift-dns/dns-default-5926b" Jul 08 21:10:45.719914 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.719892 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"image-registry-69b6b698b5-54tw8\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:10:45.758983 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.758947 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:45.814182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.814158 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-78qbx\"" Jul 08 21:10:45.814353 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.814183 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ltsxq\"" Jul 08 21:10:45.822336 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.822300 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:10:45.822427 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.822370 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5926b" Jul 08 21:10:45.887087 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:45.887057 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:46.057961 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.057926 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f4786df7f-6lrns"] Jul 08 21:10:46.061515 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.061492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.064275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064086 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pd5jn\"" Jul 08 21:10:46.064275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064095 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 21:10:46.064275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064129 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 21:10:46.064275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064097 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 21:10:46.064517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064485 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 21:10:46.064804 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.064786 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 21:10:46.069556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.069536 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 21:10:46.074115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.074087 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f4786df7f-6lrns"] Jul 08 21:10:46.221849 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.221813 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-metrics-client-ca\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222034 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.221905 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-serving-certs-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222034 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.221999 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.222080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.222119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.222183 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-federate-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.222215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zb8g\" (UniqueName: \"kubernetes.io/projected/732f43ac-236d-4bae-b9eb-304bda1cdc51-kube-api-access-8zb8g\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.222279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.222263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.322909 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.322843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.322909 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.322888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-metrics-client-ca\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323403 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.322924 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-serving-certs-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323403 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323441 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-federate-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zb8g\" (UniqueName: \"kubernetes.io/projected/732f43ac-236d-4bae-b9eb-304bda1cdc51-kube-api-access-8zb8g\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323860 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323711 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-serving-certs-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.323860 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.323807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-metrics-client-ca\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.324391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.324365 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.326014 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.325965 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.326260 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.326207 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.326260 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.326233 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-telemeter-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.326637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.326620 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/732f43ac-236d-4bae-b9eb-304bda1cdc51-federate-client-tls\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.332358 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.332337 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zb8g\" (UniqueName: \"kubernetes.io/projected/732f43ac-236d-4bae-b9eb-304bda1cdc51-kube-api-access-8zb8g\") pod \"telemeter-client-6f4786df7f-6lrns\" (UID: \"732f43ac-236d-4bae-b9eb-304bda1cdc51\") " pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:46.375217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:46.375189 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" Jul 08 21:10:47.065546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.064057 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:10:47.065546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.065503 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:47.076069 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.075574 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.084465 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.083292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 21:10:47.091038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.088975 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 21:10:47.091038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.089221 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 21:10:47.091038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.089538 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 21:10:47.091038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.089736 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-95d9d\"" Jul 08 21:10:47.091038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.090031 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.098444 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.098485 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.098877 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1iebul428qf5f\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.099080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.099375 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.099582 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.102537 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.102828 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.103034 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.103253 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 21:10:47.106120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.104269 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 21:10:47.209563 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.209532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242840 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77xh\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242888 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.242977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243001 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243026 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243092 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.243982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.245482 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.243230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.245482 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.244538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.245482 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.244587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.245482 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.244612 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.308747 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.308572 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:47.345735 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.345706 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.345766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.345795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.345813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q77xh\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346409 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346532 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346442 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346532 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346491 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346532 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346725 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346547 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346725 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346609 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346725 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.346725 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346870 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.347007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.347441 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.347728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.346760 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.348044 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.348018 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.348401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.348375 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.348950 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.348723 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.348950 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.348891 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.349110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.348981 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.349110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.349004 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.350579 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.352586 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.352635 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.353006 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353625 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.353481 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.353625 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.353515 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.354114 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.354073 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.354564 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.354156 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.354893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.354820 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.357024 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.355850 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.357771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.357726 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.359031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.358576 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77xh\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh\") pod \"prometheus-k8s-0\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.418638 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.418608 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:10:47.527446 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.527058 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:10:47.535331 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:47.535056 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029c9231_733f_43bd_b92c_17b495c2adeb.slice/crio-491344930aa0b60dc6de91ec307be96c600910a14186b752f0b9b09875a88eaf WatchSource:0}: Error finding container 491344930aa0b60dc6de91ec307be96c600910a14186b752f0b9b09875a88eaf: Status 404 returned error can't find the container with id 491344930aa0b60dc6de91ec307be96c600910a14186b752f0b9b09875a88eaf Jul 08 21:10:47.551823 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.549199 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5926b"] Jul 08 21:10:47.554115 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:47.553308 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3676d57_11c7_4391_b192_d374a9464e02.slice/crio-c20e6070ff3705592bfa747148e0052d6a781745dbca7d823b2342e4e52ccdc3 WatchSource:0}: Error finding container c20e6070ff3705592bfa747148e0052d6a781745dbca7d823b2342e4e52ccdc3: Status 404 returned error can't find the container with id c20e6070ff3705592bfa747148e0052d6a781745dbca7d823b2342e4e52ccdc3 Jul 08 21:10:47.554115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.553958 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f4786df7f-6lrns"] Jul 08 21:10:47.561190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.555975 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:10:47.585502 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.585436 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74f69594c9-k2jzc"] Jul 08 21:10:47.590216 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.590191 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:10:47.590959 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:47.590535 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0149484_2084_4ab5_9ff4_e7f737e32494.slice/crio-8c895ee403113ac2561f3cd065de759d4be3838fd07c7054a60971ba6913a2f7 WatchSource:0}: Error finding container 8c895ee403113ac2561f3cd065de759d4be3838fd07c7054a60971ba6913a2f7: Status 404 returned error can't find the container with id 8c895ee403113ac2561f3cd065de759d4be3838fd07c7054a60971ba6913a2f7 Jul 08 21:10:47.591052 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:47.590992 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b13bfd_7dd2_4c7c_9f49_ea0f9ea4f1c1.slice/crio-aad76c8675a43991153fd6fcf74c0e7b377e129ab507685e500a47885f5a44e1 WatchSource:0}: Error finding container aad76c8675a43991153fd6fcf74c0e7b377e129ab507685e500a47885f5a44e1: Status 404 returned error can't find the container with id aad76c8675a43991153fd6fcf74c0e7b377e129ab507685e500a47885f5a44e1 Jul 08 21:10:47.618177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.618145 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:10:47.625975 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:10:47.625861 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be20004_25ea_4d6d_a11e_127184e81d7c.slice/crio-d0e922f688f0121fb3d4b7f55683c4cbee1639a0e4e81c132d131edd582f5d0c WatchSource:0}: Error finding container d0e922f688f0121fb3d4b7f55683c4cbee1639a0e4e81c132d131edd582f5d0c: Status 404 returned error can't find the container with id d0e922f688f0121fb3d4b7f55683c4cbee1639a0e4e81c132d131edd582f5d0c Jul 08 21:10:47.837670 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.837583 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" event={"ID":"732f43ac-236d-4bae-b9eb-304bda1cdc51","Type":"ContainerStarted","Data":"b3310aaf3d73e37d2aa79d370a0d569510d91a1a3487252fb1c17ee9e80bc069"} Jul 08 21:10:47.839070 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.838986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76989678c5-mb7tz" event={"ID":"84299af2-b5e9-4dee-8aa7-e552c83d161e","Type":"ContainerStarted","Data":"8398fe6e33dadc5e877f49f98c345af3b2e9b51f0eb1eb397fef1d5c02253c4c"} Jul 08 21:10:47.840647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.840562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nr7" event={"ID":"113e034e-b9c8-4d67-807f-4e32c414fa3b","Type":"ContainerStarted","Data":"28bc751f27f56bfb75040cd81b3434661720095b0d3e15b752d8beaec0de649c"} Jul 08 21:10:47.842617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.842527 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ql4h7" event={"ID":"4ec95395-3efe-40f4-87bd-72afc81c739b","Type":"ContainerStarted","Data":"da4158d91c187c3b8f719cda5b7a9c955a70bb58aacbbe022617bb95f29fdcff"} Jul 08 21:10:47.844319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.844297 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:47.846102 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.846027 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"d0e922f688f0121fb3d4b7f55683c4cbee1639a0e4e81c132d131edd582f5d0c"} Jul 08 21:10:47.847957 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.847894 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5926b" event={"ID":"029c9231-733f-43bd-b92c-17b495c2adeb","Type":"ContainerStarted","Data":"491344930aa0b60dc6de91ec307be96c600910a14186b752f0b9b09875a88eaf"} Jul 08 21:10:47.849172 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.849130 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-ql4h7" Jul 08 21:10:47.852129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.852069 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" event={"ID":"c0149484-2084-4ab5-9ff4-e7f737e32494","Type":"ContainerStarted","Data":"8c895ee403113ac2561f3cd065de759d4be3838fd07c7054a60971ba6913a2f7"} Jul 08 21:10:47.854359 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.854335 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" event={"ID":"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1","Type":"ContainerStarted","Data":"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374"} Jul 08 21:10:47.854451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.854377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" event={"ID":"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1","Type":"ContainerStarted","Data":"aad76c8675a43991153fd6fcf74c0e7b377e129ab507685e500a47885f5a44e1"} Jul 08 21:10:47.854690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.854651 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:10:47.856129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.856020 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"c20e6070ff3705592bfa747148e0052d6a781745dbca7d823b2342e4e52ccdc3"} Jul 08 21:10:47.878892 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.878693 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-ql4h7" podStartSLOduration=1.281945792 podStartE2EDuration="22.8786731s" totalImagesPullingTime="21.596727308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:47.857337053 +0000 UTC m=+283.429081916" watchObservedRunningTime="2026-07-08 21:10:47.8786731 +0000 UTC m=+283.450417958" Jul 08 21:10:47.880757 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:47.880541 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" podStartSLOduration=284.880528379 podStartE2EDuration="4m44.880528379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:47.878299297 +0000 UTC m=+283.450044161" watchObservedRunningTime="2026-07-08 21:10:47.880528379 +0000 UTC m=+283.452273238" Jul 08 21:10:48.872676 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:48.871807 2565 generic.go:356] "Generic (PLEG): container finished" podID="113e034e-b9c8-4d67-807f-4e32c414fa3b" containerID="80802ee5a0874dab472594835222eb911cb8752eda94cd1b48f08f9b7698201a" exitCode=0 Jul 08 21:10:48.874285 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:48.873599 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nr7" event={"ID":"113e034e-b9c8-4d67-807f-4e32c414fa3b","Type":"ContainerDied","Data":"80802ee5a0874dab472594835222eb911cb8752eda94cd1b48f08f9b7698201a"} Jul 08 21:10:49.911365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:49.911258 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nr7" event={"ID":"113e034e-b9c8-4d67-807f-4e32c414fa3b","Type":"ContainerStarted","Data":"9dd9f6b9e062f372d85db4a774c64e737d2b940380d75eebf666b54f449767a2"} Jul 08 21:10:49.911365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:49.911333 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22nr7" event={"ID":"113e034e-b9c8-4d67-807f-4e32c414fa3b","Type":"ContainerStarted","Data":"9821d74c35c612cfb978b2863df77ed8efd5b69ef57d1e6b909a86ec2847606c"} Jul 08 21:10:49.934078 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:49.933724 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-22nr7" podStartSLOduration=8.519900793 podStartE2EDuration="9.933704789s" totalImagesPullingTime="1.307767105s" totalInitContainerRuntime="106.036891ms" isStatefulPod=true podCreationTimestamp="2026-07-08 21:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:49.930813464 +0000 UTC m=+285.502558327" watchObservedRunningTime="2026-07-08 21:10:49.933704789 +0000 UTC m=+285.505449649" Jul 08 21:10:54.945322 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.945279 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783" exitCode=0 Jul 08 21:10:54.953533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.953503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783"} Jul 08 21:10:54.953637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.953580 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5926b" event={"ID":"029c9231-733f-43bd-b92c-17b495c2adeb","Type":"ContainerStarted","Data":"b5bba5ff1943fd675cf732a63dc1de28c9a8e1d6001f23b0ce77987e53eab917"} Jul 08 21:10:54.953637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.953621 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" event={"ID":"c0149484-2084-4ab5-9ff4-e7f737e32494","Type":"ContainerStarted","Data":"8050d9a6f9a2972fd8b8884b856a8daad8b692ddbd236215324df9146848db81"} Jul 08 21:10:54.955942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.955919 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="d42346a4db8a4e4545d95788bfce68ab77a8dab88dd36fc4beca2871f9a9f31b" exitCode=0 Jul 08 21:10:54.956038 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.956026 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"d42346a4db8a4e4545d95788bfce68ab77a8dab88dd36fc4beca2871f9a9f31b"} Jul 08 21:10:54.958362 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.958292 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" event={"ID":"732f43ac-236d-4bae-b9eb-304bda1cdc51","Type":"ContainerStarted","Data":"d2c4ed14b813007fcab28a9400e654e8fa64f0dbbb837650641c5c5b41a3324e"} Jul 08 21:10:54.960784 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.960311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76989678c5-mb7tz" event={"ID":"84299af2-b5e9-4dee-8aa7-e552c83d161e","Type":"ContainerStarted","Data":"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea"} Jul 08 21:10:54.990872 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:54.990825 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76989678c5-mb7tz" podStartSLOduration=6.937111671 podStartE2EDuration="13.990806569s" totalImagesPullingTime="7.053694898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:54.990509366 +0000 UTC m=+290.562254227" watchObservedRunningTime="2026-07-08 21:10:54.990806569 +0000 UTC m=+290.562551429" Jul 08 21:10:55.037616 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.037564 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" podStartSLOduration=3.027344464 podStartE2EDuration="10.037542824s" totalImagesPullingTime="7.01019836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:55.036044176 +0000 UTC m=+290.607789038" watchObservedRunningTime="2026-07-08 21:10:55.037542824 +0000 UTC m=+290.609287684" Jul 08 21:10:55.971297 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.971253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" event={"ID":"732f43ac-236d-4bae-b9eb-304bda1cdc51","Type":"ContainerStarted","Data":"16c0f2b94a626fa8d7a74bab093aac546451fe1781b9880b1f7d4c1b42794e9c"} Jul 08 21:10:55.971854 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.971377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" event={"ID":"732f43ac-236d-4bae-b9eb-304bda1cdc51","Type":"ContainerStarted","Data":"10151fe85ff5a47791576074cecd55280209443c0acccadcfe0e53a8e51de68e"} Jul 08 21:10:55.973286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.973230 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5926b" event={"ID":"029c9231-733f-43bd-b92c-17b495c2adeb","Type":"ContainerStarted","Data":"d523afabb77a29df644e5e97d2cffc4415d25a41cb9c0d2dc5d794efdcf73920"} Jul 08 21:10:55.973418 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.973319 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5926b" Jul 08 21:10:55.994736 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:55.994688 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f4786df7f-6lrns" podStartSLOduration=2.952497496 podStartE2EDuration="9.994668401s" totalImagesPullingTime="7.042170905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:55.992539616 +0000 UTC m=+291.564284500" watchObservedRunningTime="2026-07-08 21:10:55.994668401 +0000 UTC m=+291.566413261" Jul 08 21:10:56.013275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:56.013154 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5926b" podStartSLOduration=253.961989755 podStartE2EDuration="4m21.01313818s" totalImagesPullingTime="7.051148425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:10:56.012747402 +0000 UTC m=+291.584492262" watchObservedRunningTime="2026-07-08 21:10:56.01313818 +0000 UTC m=+291.584883040" Jul 08 21:10:56.078379 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:56.078287 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:10:56.475262 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:56.475173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:56.523557 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:56.523521 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:57.149884 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:57.149147 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:57.247522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:57.247437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:57.877031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:57.876973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:57.975123 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:57.975088 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:10:58.993157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:10:58.993117 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"47af9aa9c24b84253694d0fb28ed1a1d604930b6abbc32a07b50e674543a72fa"} Jul 08 21:11:00.002773 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.002740 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1"} Jul 08 21:11:00.003226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.002815 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51"} Jul 08 21:11:00.006278 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.006256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"1a9bc4117bfce71d89f8b0f7735956df08f8b7640138caaa0916985aa7b8fa7f"} Jul 08 21:11:00.006392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.006336 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"5c79806c41f2109cc76a4ab6fcc44aace315a8699af08872d25a616019b3fcaf"} Jul 08 21:11:00.006392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.006373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"63ac007df7aa674bc8d5d96eca2e4ce6a02ee221e2ab71b4efd7b9f30476aba8"} Jul 08 21:11:00.006477 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:00.006398 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"e87f4176604925b983b61060efee8dbc60ecc9825f188daf074fbcf7ce467a89"} Jul 08 21:11:02.017968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:02.017943 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerStarted","Data":"9456f804eb7434a12d6c663c8fb7b6b6f021db779c53a325fafa715a4e924fb0"} Jul 08 21:11:02.044785 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:02.044727 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.080147152 podStartE2EDuration="21.044706668s" totalImagesPullingTime="12.943602933s" totalInitContainerRuntime="20.956583ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:11:02.041821678 +0000 UTC m=+297.613566547" watchObservedRunningTime="2026-07-08 21:11:02.044706668 +0000 UTC m=+297.616451530" Jul 08 21:11:02.164652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:02.164624 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:02.199141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:02.199109 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:11:02.264780 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:02.264752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:03.025420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:03.025390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5"} Jul 08 21:11:03.025775 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:03.025452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43"} Jul 08 21:11:03.025775 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:03.025479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8"} Jul 08 21:11:03.025775 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:03.025504 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerStarted","Data":"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3"} Jul 08 21:11:03.053527 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:03.053482 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.306775183 podStartE2EDuration="16.053468319s" totalImagesPullingTime="13.718292307s" totalInitContainerRuntime="28.400829ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:10:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:11:03.050560572 +0000 UTC m=+298.622305432" watchObservedRunningTime="2026-07-08 21:11:03.053468319 +0000 UTC m=+298.625213178" Jul 08 21:11:04.828691 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:04.828665 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:11:04.832197 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:04.832164 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:11:04.845790 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:04.845773 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 21:11:05.587093 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:05.587057 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:11:05.597193 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:05.587116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:11:05.826219 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:05.826193 2565 patch_prober.go:28] interesting pod/image-registry-69b6b698b5-54tw8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 21:11:05.826351 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:05.826252 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 21:11:05.982964 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:05.982902 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5926b" Jul 08 21:11:06.170690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:06.170656 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:07.418778 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:07.418749 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:11:07.663141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:07.663106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:07.759070 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:07.759007 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:07.959448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:07.959414 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:08.004768 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:08.004732 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:08.881518 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:08.881495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:11:09.212827 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:09.212743 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:09.312647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:09.312618 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:15.501774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:15.501730 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:15.549026 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:15.548999 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:11:15.574848 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:15.574800 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:15.627174 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:15.627147 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:15.727914 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:15.727895 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:19.092813 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:19.092779 2565 generic.go:356] "Generic (PLEG): container finished" podID="d940ecf2-11d4-4c5e-84de-87745c55deb4" containerID="3da9a7be8d5a12c60ff4027c63b1fdda9cc7f6ca2af0301b3ce5cdf7714afb79" exitCode=0 Jul 08 21:11:19.093219 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:19.092853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" event={"ID":"d940ecf2-11d4-4c5e-84de-87745c55deb4","Type":"ContainerDied","Data":"3da9a7be8d5a12c60ff4027c63b1fdda9cc7f6ca2af0301b3ce5cdf7714afb79"} Jul 08 21:11:19.093219 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:19.093140 2565 scope.go:117] "RemoveContainer" containerID="3da9a7be8d5a12c60ff4027c63b1fdda9cc7f6ca2af0301b3ce5cdf7714afb79" Jul 08 21:11:19.646149 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:19.646117 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p578w_b787178f-18f3-4d58-888d-760a65129311/serve-healthcheck-canary/0.log" Jul 08 21:11:20.099158 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:20.099131 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hrmnd" event={"ID":"d940ecf2-11d4-4c5e-84de-87745c55deb4","Type":"ContainerStarted","Data":"15b6fba1a07a58aae64f94b3fcc816cb8d966ed3f4dd9d435e9ef4d5c0ce9a00"} Jul 08 21:11:20.700032 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:20.699990 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:20.749649 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:20.749621 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:22.007403 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.007341 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76989678c5-mb7tz" podUID="84299af2-b5e9-4dee-8aa7-e552c83d161e" containerName="console" containerID="cri-o://33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea" gracePeriod=15 Jul 08 21:11:22.249911 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.249890 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:11:22.362056 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362035 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362105 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362307 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362181 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8kgx\" (UniqueName: \"kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362313 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362350 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362382 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config\") pod \"84299af2-b5e9-4dee-8aa7-e552c83d161e\" (UID: \"84299af2-b5e9-4dee-8aa7-e552c83d161e\") " Jul 08 21:11:22.362687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362659 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:22.362687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362673 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:22.362862 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362690 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca" (OuterVolumeSpecName: "service-ca") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:22.362862 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.362792 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config" (OuterVolumeSpecName: "console-config") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:22.364652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.364624 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:11:22.364723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.364658 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:11:22.364723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.364680 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx" (OuterVolumeSpecName: "kube-api-access-k8kgx") pod "84299af2-b5e9-4dee-8aa7-e552c83d161e" (UID: "84299af2-b5e9-4dee-8aa7-e552c83d161e"). InnerVolumeSpecName "kube-api-access-k8kgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:11:22.463517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463497 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-serving-cert\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463518 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-trusted-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463527 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-oauth-serving-cert\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463535 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8kgx\" (UniqueName: \"kubernetes.io/projected/84299af2-b5e9-4dee-8aa7-e552c83d161e-kube-api-access-k8kgx\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463544 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463557 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:22.463643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:22.463567 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84299af2-b5e9-4dee-8aa7-e552c83d161e-console-oauth-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:23.113182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.113150 2565 generic.go:356] "Generic (PLEG): container finished" podID="84299af2-b5e9-4dee-8aa7-e552c83d161e" containerID="33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea" exitCode=0 Jul 08 21:11:23.113567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.113201 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76989678c5-mb7tz" Jul 08 21:11:23.113567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.113246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76989678c5-mb7tz" event={"ID":"84299af2-b5e9-4dee-8aa7-e552c83d161e","Type":"ContainerDied","Data":"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea"} Jul 08 21:11:23.113567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.113299 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76989678c5-mb7tz" event={"ID":"84299af2-b5e9-4dee-8aa7-e552c83d161e","Type":"ContainerDied","Data":"8398fe6e33dadc5e877f49f98c345af3b2e9b51f0eb1eb397fef1d5c02253c4c"} Jul 08 21:11:23.113567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.113320 2565 scope.go:117] "RemoveContainer" containerID="33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea" Jul 08 21:11:23.121753 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.121734 2565 scope.go:117] "RemoveContainer" containerID="33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea" Jul 08 21:11:23.122022 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:11:23.122001 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea\": container with ID starting with 33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea not found: ID does not exist" containerID="33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea" Jul 08 21:11:23.122090 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.122029 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea"} err="failed to get container status \"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea\": rpc error: code = NotFound desc = could not find container \"33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea\": container with ID starting with 33cfcf513c9bfa913f8be969249bf10846fb13e4e2ca12ccd7bfa35b8609a5ea not found: ID does not exist" Jul 08 21:11:23.129082 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.129063 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:11:23.130458 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:23.130440 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76989678c5-mb7tz"] Jul 08 21:11:24.949161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:24.949129 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84299af2-b5e9-4dee-8aa7-e552c83d161e" path="/var/lib/kubelet/pods/84299af2-b5e9-4dee-8aa7-e552c83d161e/volumes" Jul 08 21:11:25.591599 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:25.591570 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:11:25.595259 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:25.595186 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-74f69594c9-k2jzc" Jul 08 21:11:27.216431 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:27.216399 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:27.317171 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:27.317131 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:28.137716 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:28.137683 2565 generic.go:356] "Generic (PLEG): container finished" podID="6c0aa75c-8b9a-46d8-86e9-492244a01b55" containerID="65fe0cd83a96e6c98ab742c565de8afa7c3a96b1964df589b4750664636b7dec" exitCode=0 Jul 08 21:11:28.137846 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:28.137757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" event={"ID":"6c0aa75c-8b9a-46d8-86e9-492244a01b55","Type":"ContainerDied","Data":"65fe0cd83a96e6c98ab742c565de8afa7c3a96b1964df589b4750664636b7dec"} Jul 08 21:11:28.138058 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:28.138046 2565 scope.go:117] "RemoveContainer" containerID="65fe0cd83a96e6c98ab742c565de8afa7c3a96b1964df589b4750664636b7dec" Jul 08 21:11:28.677824 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:28.677787 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:28.733004 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:28.732976 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:29.144306 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:29.144273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v7959" event={"ID":"6c0aa75c-8b9a-46d8-86e9-492244a01b55","Type":"ContainerStarted","Data":"9075245587f4163901fdeda93a27839c093da747b6f9bf9785441b923f1fcbcb"} Jul 08 21:11:29.773765 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:29.773730 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:29.826039 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:29.826004 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:32.558583 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:32.558547 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:32.606148 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:32.606115 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:40.568667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:40.568627 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerName="registry" containerID="cri-o://32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374" gracePeriod=30 Jul 08 21:11:40.840782 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:40.840730 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:11:41.014998 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.014973 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015056 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4svj\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015096 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015125 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015164 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015211 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015234 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015281 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates\") pod \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\" (UID: \"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1\") " Jul 08 21:11:41.015795 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015637 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:41.015909 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.015878 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:11:41.017756 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.017728 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:11:41.017880 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.017780 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:11:41.017880 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.017794 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj" (OuterVolumeSpecName: "kube-api-access-n4svj") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "kube-api-access-n4svj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:11:41.017880 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.017854 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:11:41.018049 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.017918 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:11:41.023277 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.023223 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" (UID: "43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:11:41.116449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116398 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4svj\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-kube-api-access-n4svj\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116420 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-bound-sa-token\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116430 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-ca-trust-extracted\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116438 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-trusted-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116447 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-image-registry-private-configuration\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116457 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116465 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-registry-certificates\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.116653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.116473 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1-installation-pull-secrets\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:11:41.207180 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.207145 2565 generic.go:356] "Generic (PLEG): container finished" podID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerID="32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374" exitCode=0 Jul 08 21:11:41.207341 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.207185 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" Jul 08 21:11:41.207341 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.207200 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" event={"ID":"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1","Type":"ContainerDied","Data":"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374"} Jul 08 21:11:41.207341 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.207265 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b6b698b5-54tw8" event={"ID":"43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1","Type":"ContainerDied","Data":"aad76c8675a43991153fd6fcf74c0e7b377e129ab507685e500a47885f5a44e1"} Jul 08 21:11:41.207341 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.207297 2565 scope.go:117] "RemoveContainer" containerID="32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374" Jul 08 21:11:41.216511 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.216490 2565 scope.go:117] "RemoveContainer" containerID="32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374" Jul 08 21:11:41.216915 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:11:41.216836 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374\": container with ID starting with 32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374 not found: ID does not exist" containerID="32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374" Jul 08 21:11:41.216915 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.216878 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374"} err="failed to get container status \"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374\": rpc error: code = NotFound desc = could not find container \"32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374\": container with ID starting with 32851bf46e9d609ef233d8ad2566b18b1195cd5e8d4c8185653d0f6d187ce374 not found: ID does not exist" Jul 08 21:11:41.236766 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.236711 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:11:41.238672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:41.238648 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69b6b698b5-54tw8"] Jul 08 21:11:42.949338 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:42.949308 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" path="/var/lib/kubelet/pods/43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1/volumes" Jul 08 21:11:43.682623 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:43.682592 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:43.731758 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:43.731730 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:47.419639 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:47.419610 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:11:47.442986 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:47.442959 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:11:48.255116 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:48.255091 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:11:49.830817 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:49.830773 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:49.931678 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:49.931646 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:53.515840 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:53.515808 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:53.581894 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:53.581861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:56.916394 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:56.916313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:11:56.975110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:11:56.975082 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:01.154622 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154591 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:01.155024 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154847 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="alertmanager" containerID="cri-o://47af9aa9c24b84253694d0fb28ed1a1d604930b6abbc32a07b50e674543a72fa" gracePeriod=120 Jul 08 21:12:01.155024 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154915 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-metric" containerID="cri-o://1a9bc4117bfce71d89f8b0f7735956df08f8b7640138caaa0916985aa7b8fa7f" gracePeriod=120 Jul 08 21:12:01.155024 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154960 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-web" containerID="cri-o://63ac007df7aa674bc8d5d96eca2e4ce6a02ee221e2ab71b4efd7b9f30476aba8" gracePeriod=120 Jul 08 21:12:01.155257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154996 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy" containerID="cri-o://5c79806c41f2109cc76a4ab6fcc44aace315a8699af08872d25a616019b3fcaf" gracePeriod=120 Jul 08 21:12:01.155257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.155023 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="prom-label-proxy" containerID="cri-o://9456f804eb7434a12d6c663c8fb7b6b6f021db779c53a325fafa715a4e924fb0" gracePeriod=120 Jul 08 21:12:01.155257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.154994 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="config-reloader" containerID="cri-o://e87f4176604925b983b61060efee8dbc60ecc9825f188daf074fbcf7ce467a89" gracePeriod=120 Jul 08 21:12:01.296064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296039 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="9456f804eb7434a12d6c663c8fb7b6b6f021db779c53a325fafa715a4e924fb0" exitCode=0 Jul 08 21:12:01.296064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296061 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="1a9bc4117bfce71d89f8b0f7735956df08f8b7640138caaa0916985aa7b8fa7f" exitCode=0 Jul 08 21:12:01.296064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296067 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="5c79806c41f2109cc76a4ab6fcc44aace315a8699af08872d25a616019b3fcaf" exitCode=0 Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296073 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="e87f4176604925b983b61060efee8dbc60ecc9825f188daf074fbcf7ce467a89" exitCode=0 Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296078 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="47af9aa9c24b84253694d0fb28ed1a1d604930b6abbc32a07b50e674543a72fa" exitCode=0 Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296106 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"9456f804eb7434a12d6c663c8fb7b6b6f021db779c53a325fafa715a4e924fb0"} Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"1a9bc4117bfce71d89f8b0f7735956df08f8b7640138caaa0916985aa7b8fa7f"} Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"5c79806c41f2109cc76a4ab6fcc44aace315a8699af08872d25a616019b3fcaf"} Jul 08 21:12:01.296224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"e87f4176604925b983b61060efee8dbc60ecc9825f188daf074fbcf7ce467a89"} Jul 08 21:12:01.296445 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:01.296258 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"47af9aa9c24b84253694d0fb28ed1a1d604930b6abbc32a07b50e674543a72fa"} Jul 08 21:12:02.303902 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.303870 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3676d57-11c7-4391-b192-d374a9464e02" containerID="63ac007df7aa674bc8d5d96eca2e4ce6a02ee221e2ab71b4efd7b9f30476aba8" exitCode=0 Jul 08 21:12:02.304268 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.303931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"63ac007df7aa674bc8d5d96eca2e4ce6a02ee221e2ab71b4efd7b9f30476aba8"} Jul 08 21:12:02.405532 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.405505 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:02.471605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471537 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8p8t\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471575 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471595 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471696 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471724 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471768 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471791 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.471838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471822 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471853 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471902 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471932 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471959 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config\") pod \"a3676d57-11c7-4391-b192-d374a9464e02\" (UID: \"a3676d57-11c7-4391-b192-d374a9464e02\") " Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.471961 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:12:02.472141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.472041 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:02.472535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.472270 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-main-db\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.472535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.472289 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-metrics-client-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.472651 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.472625 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:02.474956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.474927 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t" (OuterVolumeSpecName: "kube-api-access-c8p8t") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "kube-api-access-c8p8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:12:02.475169 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.475146 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume" (OuterVolumeSpecName: "config-volume") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.475656 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.475630 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.475656 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.475631 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:12:02.475812 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.475768 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.476388 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.476360 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.476737 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.476693 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.476855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.476755 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out" (OuterVolumeSpecName: "config-out") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:12:02.479685 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.479657 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.487713 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.487691 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config" (OuterVolumeSpecName: "web-config") pod "a3676d57-11c7-4391-b192-d374a9464e02" (UID: "a3676d57-11c7-4391-b192-d374a9464e02"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:02.573230 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573206 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8p8t\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-kube-api-access-c8p8t\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573230 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573228 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-config-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573262 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573277 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573291 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-main-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573303 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3676d57-11c7-4391-b192-d374a9464e02-config-out\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573312 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573338 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-web-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573347 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3676d57-11c7-4391-b192-d374a9464e02-tls-assets\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573354 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3676d57-11c7-4391-b192-d374a9464e02-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:02.573585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:02.573363 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3676d57-11c7-4391-b192-d374a9464e02-cluster-tls-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:03.311271 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.311226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3676d57-11c7-4391-b192-d374a9464e02","Type":"ContainerDied","Data":"c20e6070ff3705592bfa747148e0052d6a781745dbca7d823b2342e4e52ccdc3"} Jul 08 21:12:03.311590 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.311279 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.311590 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.311330 2565 scope.go:117] "RemoveContainer" containerID="9456f804eb7434a12d6c663c8fb7b6b6f021db779c53a325fafa715a4e924fb0" Jul 08 21:12:03.318922 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.318903 2565 scope.go:117] "RemoveContainer" containerID="1a9bc4117bfce71d89f8b0f7735956df08f8b7640138caaa0916985aa7b8fa7f" Jul 08 21:12:03.325489 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.325475 2565 scope.go:117] "RemoveContainer" containerID="5c79806c41f2109cc76a4ab6fcc44aace315a8699af08872d25a616019b3fcaf" Jul 08 21:12:03.332458 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.332422 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:03.334130 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.334115 2565 scope.go:117] "RemoveContainer" containerID="63ac007df7aa674bc8d5d96eca2e4ce6a02ee221e2ab71b4efd7b9f30476aba8" Jul 08 21:12:03.334377 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.334360 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:03.340592 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.340576 2565 scope.go:117] "RemoveContainer" containerID="e87f4176604925b983b61060efee8dbc60ecc9825f188daf074fbcf7ce467a89" Jul 08 21:12:03.346935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.346920 2565 scope.go:117] "RemoveContainer" containerID="47af9aa9c24b84253694d0fb28ed1a1d604930b6abbc32a07b50e674543a72fa" Jul 08 21:12:03.353771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.353756 2565 scope.go:117] "RemoveContainer" containerID="d42346a4db8a4e4545d95788bfce68ab77a8dab88dd36fc4beca2871f9a9f31b" Jul 08 21:12:03.358192 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358172 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:03.358519 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358504 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-web" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358522 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-web" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358531 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="init-config-reloader" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358537 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="init-config-reloader" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358543 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="config-reloader" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358549 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="config-reloader" Jul 08 21:12:03.358560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358559 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-metric" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358564 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-metric" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358577 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358582 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358591 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84299af2-b5e9-4dee-8aa7-e552c83d161e" containerName="console" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358597 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="84299af2-b5e9-4dee-8aa7-e552c83d161e" containerName="console" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358605 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="prom-label-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358610 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="prom-label-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358616 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerName="registry" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358621 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerName="registry" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358629 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="alertmanager" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358634 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="alertmanager" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358696 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="84299af2-b5e9-4dee-8aa7-e552c83d161e" containerName="console" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358704 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="prom-label-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358711 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-metric" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358717 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43b13bfd-7dd2-4c7c-9f49-ea0f9ea4f1c1" containerName="registry" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358722 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="config-reloader" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358727 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358732 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="kube-rbac-proxy-web" Jul 08 21:12:03.358764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.358737 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3676d57-11c7-4391-b192-d374a9464e02" containerName="alertmanager" Jul 08 21:12:03.362216 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.362197 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.364600 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364584 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 21:12:03.364687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364669 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 21:12:03.364964 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364910 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 21:12:03.364964 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364954 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 21:12:03.365106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364982 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 21:12:03.365106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.364985 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 21:12:03.365106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.365005 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 21:12:03.365106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.365058 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 21:12:03.365106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.365098 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k6htb\"" Jul 08 21:12:03.371128 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.371104 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 21:12:03.378020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.377997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-out\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378032 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378370 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378335 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8n5v\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-kube-api-access-t8n5v\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378569 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-web-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.378762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.378712 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:03.478891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.478841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-out\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.478891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.478869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.478891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.478888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.478904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479084 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479084 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479109 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479177 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479166 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479333 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479333 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479267 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8n5v\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-kube-api-access-t8n5v\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479333 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.479503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-web-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.480971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479733 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.480971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.479867 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.480971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.480387 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b2c8d16-aa82-4462-b9f8-78176a1255bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.481651 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.481623 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-out\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.481880 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.481894 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.482017 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-web-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.482105 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.482369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.482450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.482425 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.483589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.483569 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.484052 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.484033 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b2c8d16-aa82-4462-b9f8-78176a1255bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.487137 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.487116 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8n5v\" (UniqueName: \"kubernetes.io/projected/6b2c8d16-aa82-4462-b9f8-78176a1255bf-kube-api-access-t8n5v\") pod \"alertmanager-main-0\" (UID: \"6b2c8d16-aa82-4462-b9f8-78176a1255bf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.677503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.677479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:12:03.809711 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:03.809684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:12:03.811409 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:12:03.811382 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2c8d16_aa82_4462_b9f8_78176a1255bf.slice/crio-8430423233d5651edc8e3d44918cf300b0712ed4ada518661ef9ea1a28e4f1f7 WatchSource:0}: Error finding container 8430423233d5651edc8e3d44918cf300b0712ed4ada518661ef9ea1a28e4f1f7: Status 404 returned error can't find the container with id 8430423233d5651edc8e3d44918cf300b0712ed4ada518661ef9ea1a28e4f1f7 Jul 08 21:12:04.317376 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:04.317342 2565 generic.go:356] "Generic (PLEG): container finished" podID="6b2c8d16-aa82-4462-b9f8-78176a1255bf" containerID="073cfebfe5744396c0e45dbe420e3093dc6f732de77b3828c2456619d83836a7" exitCode=0 Jul 08 21:12:04.317774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:04.317426 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerDied","Data":"073cfebfe5744396c0e45dbe420e3093dc6f732de77b3828c2456619d83836a7"} Jul 08 21:12:04.317774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:04.317521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"8430423233d5651edc8e3d44918cf300b0712ed4ada518661ef9ea1a28e4f1f7"} Jul 08 21:12:04.949741 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:04.949717 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3676d57-11c7-4391-b192-d374a9464e02" path="/var/lib/kubelet/pods/a3676d57-11c7-4391-b192-d374a9464e02/volumes" Jul 08 21:12:05.221137 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.221052 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:05.326259 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.323064 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:05.332139 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332101 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"567b85107f869fb08c85a886186287b8a7eb55a18108a2f596ae76dc0ec410e0"} Jul 08 21:12:05.332271 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"be1eea1aa0b30bf6eea6c1c2d64b8327c894e2bebd30aa8991a2acb6ba6d80e1"} Jul 08 21:12:05.332320 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332276 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"d74e62b7e849b9ec9197cdab5a01c1348db7696f4b59a71889639e3a572a3c58"} Jul 08 21:12:05.332320 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"7022cf43dd399bbc682e2759a2cc26eb46f63d6e0d5052e4532c1afc84738ba6"} Jul 08 21:12:05.332381 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332354 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"b5a671b65f797c7e86c73a2e1490715f74dc3b9ab1889e2d48e08b31a2ee2292"} Jul 08 21:12:05.332418 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.332397 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b2c8d16-aa82-4462-b9f8-78176a1255bf","Type":"ContainerStarted","Data":"118ac1a89da078956b2b68d1b4847623fd616b37afe2e8bd048004c48e8c6774"} Jul 08 21:12:05.359461 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.359409 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.343566543 podStartE2EDuration="2.35939112s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.824577ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:12:05.356842565 +0000 UTC m=+360.928587424" watchObservedRunningTime="2026-07-08 21:12:05.35939112 +0000 UTC m=+360.931135981" Jul 08 21:12:05.411973 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.411940 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:05.412380 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412325 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="prometheus" containerID="cri-o://74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51" gracePeriod=600 Jul 08 21:12:05.412496 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412382 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5" gracePeriod=600 Jul 08 21:12:05.412496 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412415 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="thanos-sidecar" containerID="cri-o://3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3" gracePeriod=600 Jul 08 21:12:05.412617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412496 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="config-reloader" containerID="cri-o://3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1" gracePeriod=600 Jul 08 21:12:05.412617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412522 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-web" containerID="cri-o://532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8" gracePeriod=600 Jul 08 21:12:05.412617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.412341 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy" containerID="cri-o://b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43" gracePeriod=600 Jul 08 21:12:05.413350 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.413033 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:05.963831 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:05.963790 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:06.010186 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.010157 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:06.339920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339893 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5" exitCode=0 Jul 08 21:12:06.339920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339914 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43" exitCode=0 Jul 08 21:12:06.339920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339921 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3" exitCode=0 Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339928 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1" exitCode=0 Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339933 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51" exitCode=0 Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.339961 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5"} Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.340023 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43"} Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.340058 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3"} Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.340084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1"} Jul 08 21:12:06.340299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.340108 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51"} Jul 08 21:12:06.672938 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.672916 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:06.704081 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704059 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704088 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q77xh\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704115 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704141 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704184 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704221 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704269 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704297 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704325 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704364 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704395 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704434 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704460 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704496 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704517 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704501 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704528 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704831 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704884 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704913 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704914 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704946 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704945 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.704999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.704990 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy\") pod \"9be20004-25ea-4d6d-a11e-127184e81d7c\" (UID: \"9be20004-25ea-4d6d-a11e-127184e81d7c\") " Jul 08 21:12:06.705397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.705327 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.705397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.705351 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.705397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.705366 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.705986 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.705956 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.706451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.706425 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.706617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.706591 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.707539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.707423 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:12:06.708951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.708909 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out" (OuterVolumeSpecName: "config-out") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:12:06.710271 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.710208 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.710798 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.710770 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:12:06.711706 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.711676 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.711830 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.711783 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.711893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.711870 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:12:06.711945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.711919 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.712048 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.712019 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.712112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.712088 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.712112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.712100 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config" (OuterVolumeSpecName: "config") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.712410 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.712387 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh" (OuterVolumeSpecName: "kube-api-access-q77xh") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "kube-api-access-q77xh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:12:06.713891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.713871 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.728454 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.728431 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config" (OuterVolumeSpecName: "web-config") pod "9be20004-25ea-4d6d-a11e-127184e81d7c" (UID: "9be20004-25ea-4d6d-a11e-127184e81d7c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805774 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-metrics-client-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805807 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805817 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-tls-assets\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805827 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-web-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805835 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.805833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805844 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805852 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-config-out\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805860 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805868 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-db\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805877 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9be20004-25ea-4d6d-a11e-127184e81d7c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805885 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-metrics-client-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805894 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805903 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-grpc-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805911 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-kube-rbac-proxy\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805919 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805928 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q77xh\" (UniqueName: \"kubernetes.io/projected/9be20004-25ea-4d6d-a11e-127184e81d7c-kube-api-access-q77xh\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:06.806054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:06.805936 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9be20004-25ea-4d6d-a11e-127184e81d7c-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:12:07.328220 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.328191 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:07.347784 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.347760 2565 generic.go:356] "Generic (PLEG): container finished" podID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerID="532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8" exitCode=0 Jul 08 21:12:07.348067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.347810 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8"} Jul 08 21:12:07.348067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.347830 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.348067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.347885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9be20004-25ea-4d6d-a11e-127184e81d7c","Type":"ContainerDied","Data":"d0e922f688f0121fb3d4b7f55683c4cbee1639a0e4e81c132d131edd582f5d0c"} Jul 08 21:12:07.348067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.347919 2565 scope.go:117] "RemoveContainer" containerID="e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5" Jul 08 21:12:07.355217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.355195 2565 scope.go:117] "RemoveContainer" containerID="b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43" Jul 08 21:12:07.361878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.361863 2565 scope.go:117] "RemoveContainer" containerID="532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8" Jul 08 21:12:07.368069 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.368048 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:07.369198 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.369182 2565 scope.go:117] "RemoveContainer" containerID="3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3" Jul 08 21:12:07.371612 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.371595 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:07.375927 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.375912 2565 scope.go:117] "RemoveContainer" containerID="3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1" Jul 08 21:12:07.382424 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.382410 2565 scope.go:117] "RemoveContainer" containerID="74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51" Jul 08 21:12:07.388467 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.388443 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:07.391407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.391390 2565 scope.go:117] "RemoveContainer" containerID="d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783" Jul 08 21:12:07.398410 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.398333 2565 scope.go:117] "RemoveContainer" containerID="e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5" Jul 08 21:12:07.398659 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.398640 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5\": container with ID starting with e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5 not found: ID does not exist" containerID="e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5" Jul 08 21:12:07.398745 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.398666 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5"} err="failed to get container status \"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5\": rpc error: code = NotFound desc = could not find container \"e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5\": container with ID starting with e5156bf09a5c6f0bf325d3f50f9970ebe44975a133abeb8b4c9c5ede672e4ae5 not found: ID does not exist" Jul 08 21:12:07.398745 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.398683 2565 scope.go:117] "RemoveContainer" containerID="b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43" Jul 08 21:12:07.399016 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.398959 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43\": container with ID starting with b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43 not found: ID does not exist" containerID="b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43" Jul 08 21:12:07.399064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399016 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43"} err="failed to get container status \"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43\": rpc error: code = NotFound desc = could not find container \"b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43\": container with ID starting with b6ce107690079384bc9c983bc8530327cad8477279fc84adacb1f74d52698f43 not found: ID does not exist" Jul 08 21:12:07.399064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399038 2565 scope.go:117] "RemoveContainer" containerID="532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8" Jul 08 21:12:07.399345 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.399323 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8\": container with ID starting with 532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8 not found: ID does not exist" containerID="532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8" Jul 08 21:12:07.399441 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399353 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8"} err="failed to get container status \"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8\": rpc error: code = NotFound desc = could not find container \"532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8\": container with ID starting with 532391a5ef0dadfe759bd70304e03279c8c47eb7e51082511800152fab1befa8 not found: ID does not exist" Jul 08 21:12:07.399441 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399374 2565 scope.go:117] "RemoveContainer" containerID="3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3" Jul 08 21:12:07.399676 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.399653 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3\": container with ID starting with 3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3 not found: ID does not exist" containerID="3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3" Jul 08 21:12:07.399749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399686 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3"} err="failed to get container status \"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3\": rpc error: code = NotFound desc = could not find container \"3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3\": container with ID starting with 3876bd4627f178d54dd8c6b25d0da0c31824983c02d957c437d486828b38fff3 not found: ID does not exist" Jul 08 21:12:07.399749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399701 2565 scope.go:117] "RemoveContainer" containerID="3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1" Jul 08 21:12:07.399942 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.399925 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1\": container with ID starting with 3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1 not found: ID does not exist" containerID="3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1" Jul 08 21:12:07.399999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399950 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1"} err="failed to get container status \"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1\": rpc error: code = NotFound desc = could not find container \"3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1\": container with ID starting with 3a7ad0c5ebc9ff316dae2823aff5a03a851f71b205cbda4c76fb9603250481e1 not found: ID does not exist" Jul 08 21:12:07.399999 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.399971 2565 scope.go:117] "RemoveContainer" containerID="74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51" Jul 08 21:12:07.400183 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.400165 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51\": container with ID starting with 74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51 not found: ID does not exist" containerID="74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51" Jul 08 21:12:07.400262 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400188 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51"} err="failed to get container status \"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51\": rpc error: code = NotFound desc = could not find container \"74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51\": container with ID starting with 74520f254e71fc5df1fb0596e926bdae89078edc57bd8010426005553f822e51 not found: ID does not exist" Jul 08 21:12:07.400262 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400207 2565 scope.go:117] "RemoveContainer" containerID="d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783" Jul 08 21:12:07.400434 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:12:07.400419 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783\": container with ID starting with d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783 not found: ID does not exist" containerID="d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783" Jul 08 21:12:07.400470 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400437 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783"} err="failed to get container status \"d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783\": rpc error: code = NotFound desc = could not find container \"d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783\": container with ID starting with d18e3b6284cd3c1e727692e9bb4d85090009eb3f00339415d67ebe3a92738783 not found: ID does not exist" Jul 08 21:12:07.400502 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400486 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:07.400772 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400761 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy" Jul 08 21:12:07.400809 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400774 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy" Jul 08 21:12:07.400809 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400789 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="thanos-sidecar" Jul 08 21:12:07.400809 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400794 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="thanos-sidecar" Jul 08 21:12:07.400809 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400802 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-thanos" Jul 08 21:12:07.400809 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400807 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-thanos" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400814 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="init-config-reloader" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400820 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="init-config-reloader" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400826 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-web" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400832 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-web" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400839 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="config-reloader" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400844 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="config-reloader" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400851 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="prometheus" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400863 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="prometheus" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400907 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-web" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400914 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400921 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="prometheus" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400927 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="config-reloader" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400933 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="thanos-sidecar" Jul 08 21:12:07.400942 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.400940 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" containerName="kube-rbac-proxy-thanos" Jul 08 21:12:07.411230 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.411210 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.414159 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414138 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-95d9d\"" Jul 08 21:12:07.414280 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414158 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1iebul428qf5f\"" Jul 08 21:12:07.414280 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414141 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 21:12:07.414409 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414292 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 21:12:07.414533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414463 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 21:12:07.414533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414493 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 21:12:07.414733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414535 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 21:12:07.414733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414550 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 21:12:07.414733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414660 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 21:12:07.414733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414466 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 21:12:07.414733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414732 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 21:12:07.415030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414778 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 21:12:07.415030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414791 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 21:12:07.415030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.414969 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 21:12:07.420070 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.420021 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 21:12:07.422584 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.422566 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:07.423225 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.423202 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 21:12:07.511057 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511057 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511057 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511201 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511199 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnfqx\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-kube-api-access-xnfqx\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511231 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511312 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511365 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511424 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.511572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.511460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612188 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612165 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnfqx\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-kube-api-access-xnfqx\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612430 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612509 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.612663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.613046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.613046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612704 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.613046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.612985 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.613046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.613010 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.613279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.613199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615111 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.614158 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615111 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.614817 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615669 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.615375 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615669 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.615399 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615826 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.615743 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.615870 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.615950 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.615891 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.616312 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.616290 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.616392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.616298 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.616392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.616383 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.616728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.616706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.617311 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.617286 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.617919 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.617901 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.618170 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.618153 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.618357 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.618328 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.618617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.618598 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.620042 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.620025 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnfqx\" (UniqueName: \"kubernetes.io/projected/8fc5e2ea-ca74-47f8-a8b6-7df455e0b278-kube-api-access-xnfqx\") pod \"prometheus-k8s-0\" (UID: \"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.723893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.723867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:07.854701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:07.854675 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:12:07.857645 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:12:07.857622 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc5e2ea_ca74_47f8_a8b6_7df455e0b278.slice/crio-10434ef8a4f3bebfa1ec0e0ed9f7b8abbe0669279d732114b79edf221e5fca73 WatchSource:0}: Error finding container 10434ef8a4f3bebfa1ec0e0ed9f7b8abbe0669279d732114b79edf221e5fca73: Status 404 returned error can't find the container with id 10434ef8a4f3bebfa1ec0e0ed9f7b8abbe0669279d732114b79edf221e5fca73 Jul 08 21:12:08.354322 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:08.354278 2565 generic.go:356] "Generic (PLEG): container finished" podID="8fc5e2ea-ca74-47f8-a8b6-7df455e0b278" containerID="89af0673924b52428d58aa38b941a2523090c264eab571de1f0f24edfb25a9b9" exitCode=0 Jul 08 21:12:08.354609 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:08.354368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerDied","Data":"89af0673924b52428d58aa38b941a2523090c264eab571de1f0f24edfb25a9b9"} Jul 08 21:12:08.354609 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:08.354436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"10434ef8a4f3bebfa1ec0e0ed9f7b8abbe0669279d732114b79edf221e5fca73"} Jul 08 21:12:08.949259 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:08.949184 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9be20004-25ea-4d6d-a11e-127184e81d7c" path="/var/lib/kubelet/pods/9be20004-25ea-4d6d-a11e-127184e81d7c/volumes" Jul 08 21:12:09.362161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362133 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"b6a2f0b2d1fbbadba8b8178d14cdb5dbf925348c3fcd760a24decbc0275a5e30"} Jul 08 21:12:09.362485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"a31427322ff0920164ce45be407e571d3925d856413c4e1b6d30767f78bf7ae6"} Jul 08 21:12:09.362485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"1cc6fd493120094ed022b9196bc905f60eaa972751afb13687eb584d12071463"} Jul 08 21:12:09.362485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"644e469e917666a17dd182b6bece3d51638e4d0d1da69f294d8c31f1259035e2"} Jul 08 21:12:09.362485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362331 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"55b08a8c47fba12d55dc1e0a5a7a0d094927f69e9a2216526cdf63f9f4a81bbe"} Jul 08 21:12:09.362485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.362357 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fc5e2ea-ca74-47f8-a8b6-7df455e0b278","Type":"ContainerStarted","Data":"dde5b7de467b883009afccd3cb477d12b45eab0b22a4523551b3c4b37339c661"} Jul 08 21:12:09.391268 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:09.390860 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.374481896 podStartE2EDuration="2.390840544s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.358648ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:12:09.387468054 +0000 UTC m=+364.959212947" watchObservedRunningTime="2026-07-08 21:12:09.390840544 +0000 UTC m=+364.962585404" Jul 08 21:12:10.835478 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:10.835446 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:12.724027 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:12.723999 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:12:13.689956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:13.689919 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:13.787887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:13.787850 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:16.545671 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:16.545620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:16.595095 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:16.595061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:16.649361 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:16.649334 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:18.512923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:18.512885 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:18.566835 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:18.566805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:49.867967 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:49.867927 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:56.138331 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:56.138284 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:12:56.237113 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:12:56.237087 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:07.724946 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:07.724913 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:13:07.739764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:07.739741 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:13:08.618071 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:08.618047 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:13:22.687083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:22.687044 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:23.476834 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:23.476798 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:26.294761 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:26.294687 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:26.353136 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:26.353109 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:31.689550 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:31.689508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:31.733977 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:31.733952 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:35.118940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:35.118907 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:35.216645 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:35.216612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:37.480564 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:37.480521 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:37.680714 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:37.680681 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:42.585115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:42.585065 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:42.634485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:42.634456 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:45.075794 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:45.075766 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:13:46.993687 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:46.993654 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6dd7v"] Jul 08 21:13:46.996743 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:46.996729 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:46.999034 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:46.999007 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 21:13:46.999987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:46.999968 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 21:13:47.000082 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.000031 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7x7t4\"" Jul 08 21:13:47.003332 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.003312 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6dd7v"] Jul 08 21:13:47.122268 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.122226 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwv5v\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-kube-api-access-xwv5v\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.122384 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.122358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-bound-sa-token\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.222906 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.222880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-bound-sa-token\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.223032 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.222924 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwv5v\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-kube-api-access-xwv5v\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.230147 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.230123 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-bound-sa-token\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.230437 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.230417 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwv5v\" (UniqueName: \"kubernetes.io/projected/bc53430a-fa3c-491d-b30a-c245cbab5a9d-kube-api-access-xwv5v\") pod \"cert-manager-55b4d954d5-6dd7v\" (UID: \"bc53430a-fa3c-491d-b30a-c245cbab5a9d\") " pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.314343 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.314314 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6dd7v" Jul 08 21:13:47.439537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.439513 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6dd7v"] Jul 08 21:13:47.441715 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:13:47.441683 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc53430a_fa3c_491d_b30a_c245cbab5a9d.slice/crio-af4081ca04e914c6df931ab95582187287cd40595d7ae3edc565dd0e5d0df8d8 WatchSource:0}: Error finding container af4081ca04e914c6df931ab95582187287cd40595d7ae3edc565dd0e5d0df8d8: Status 404 returned error can't find the container with id af4081ca04e914c6df931ab95582187287cd40595d7ae3edc565dd0e5d0df8d8 Jul 08 21:13:47.443591 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.443572 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:13:47.758432 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:47.758368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6dd7v" event={"ID":"bc53430a-fa3c-491d-b30a-c245cbab5a9d","Type":"ContainerStarted","Data":"af4081ca04e914c6df931ab95582187287cd40595d7ae3edc565dd0e5d0df8d8"} Jul 08 21:13:50.774420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:50.774347 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6dd7v" event={"ID":"bc53430a-fa3c-491d-b30a-c245cbab5a9d","Type":"ContainerStarted","Data":"84838b56c1fa08bbf4e3a2595ef13d6b96b93448890c74e591029c22fb170c69"} Jul 08 21:13:50.790255 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:13:50.790202 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6dd7v" podStartSLOduration=1.756530696 podStartE2EDuration="4.790188288s" totalImagesPullingTime="3.033657592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:13:50.788508697 +0000 UTC m=+466.360253553" watchObservedRunningTime="2026-07-08 21:13:50.790188288 +0000 UTC m=+466.361933147" Jul 08 21:14:03.623749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:03.623705 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:03.722174 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:03.722147 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:05.263317 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.263288 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw"] Jul 08 21:14:05.266404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.266389 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.268632 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.268616 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:14:05.269602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.269585 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vjmnk\"" Jul 08 21:14:05.269667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.269593 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 21:14:05.273672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.273651 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw"] Jul 08 21:14:05.346743 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.346718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdv7r\" (UniqueName: \"kubernetes.io/projected/7e508b52-c8b3-4445-b142-62267ea72e8a-kube-api-access-rdv7r\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.346837 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.346760 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e508b52-c8b3-4445-b142-62267ea72e8a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.447258 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.447216 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdv7r\" (UniqueName: \"kubernetes.io/projected/7e508b52-c8b3-4445-b142-62267ea72e8a-kube-api-access-rdv7r\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.447354 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.447283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e508b52-c8b3-4445-b142-62267ea72e8a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.447606 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.447587 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e508b52-c8b3-4445-b142-62267ea72e8a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.455661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.455642 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdv7r\" (UniqueName: \"kubernetes.io/projected/7e508b52-c8b3-4445-b142-62267ea72e8a-kube-api-access-rdv7r\") pod \"openshift-lws-operator-fd8ccff4c-x25sw\" (UID: \"7e508b52-c8b3-4445-b142-62267ea72e8a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.576035 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.576007 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" Jul 08 21:14:05.705730 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.703751 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw"] Jul 08 21:14:05.707284 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:05.707254 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e508b52_c8b3_4445_b142_62267ea72e8a.slice/crio-9177e3d3d9fdedd273fa8d7059ae9a07ed2dde2dd32a8ff3c7d3f8437f90e00f WatchSource:0}: Error finding container 9177e3d3d9fdedd273fa8d7059ae9a07ed2dde2dd32a8ff3c7d3f8437f90e00f: Status 404 returned error can't find the container with id 9177e3d3d9fdedd273fa8d7059ae9a07ed2dde2dd32a8ff3c7d3f8437f90e00f Jul 08 21:14:05.835886 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:05.835831 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" event={"ID":"7e508b52-c8b3-4445-b142-62267ea72e8a","Type":"ContainerStarted","Data":"9177e3d3d9fdedd273fa8d7059ae9a07ed2dde2dd32a8ff3c7d3f8437f90e00f"} Jul 08 21:14:08.447674 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:08.447631 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:08.493346 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:08.493314 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:08.614262 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:08.614209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:08.850227 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:08.850201 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" event={"ID":"7e508b52-c8b3-4445-b142-62267ea72e8a","Type":"ContainerStarted","Data":"048c9dc0835e26453b19f38d7bd78c3a3554b01b39c0ae452790c52e988bfefa"} Jul 08 21:14:08.865979 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:08.865928 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x25sw" podStartSLOduration=0.805252003 podStartE2EDuration="3.865913116s" totalImagesPullingTime="3.060661113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:08.863775845 +0000 UTC m=+484.435520707" watchObservedRunningTime="2026-07-08 21:14:08.865913116 +0000 UTC m=+484.437657974" Jul 08 21:14:09.554507 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:09.554467 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:09.654662 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:09.654623 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:11.503648 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:11.503614 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:23.791191 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:23.791145 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:23.875660 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:23.875630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:23.980351 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:23.980322 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:24.097962 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.097891 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:24.602760 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.602733 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:24.802257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.802207 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:24.975853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.975788 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4"] Jul 08 21:14:24.980067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.980052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:24.982912 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.982891 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 21:14:24.983108 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.983083 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 21:14:24.983206 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.983168 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 21:14:24.983282 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.983203 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w5666\"" Jul 08 21:14:24.983335 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:24.983312 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 21:14:25.002018 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.002000 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4"] Jul 08 21:14:25.097300 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.097272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pff79\" (UniqueName: \"kubernetes.io/projected/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-kube-api-access-pff79\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.097404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.097318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.097404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.097386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-tmp\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.097493 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.097452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.097551 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.097500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-manifests\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.111664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.111639 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.198039 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.198127 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-tmp\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.198127 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198108 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.198229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-manifests\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.198229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198167 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pff79\" (UniqueName: \"kubernetes.io/projected/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-kube-api-access-pff79\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.198573 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.198545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-manifests\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.200309 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.200282 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-tmp\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.200561 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.200538 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.200661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.200562 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.201596 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.201576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.205163 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.205144 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pff79\" (UniqueName: \"kubernetes.io/projected/e315be0a-8872-40ce-b8c4-c0c1285a4bfa-kube-api-access-pff79\") pod \"opendatahub-operator-controller-manager-94bc986b9-dcpp4\" (UID: \"e315be0a-8872-40ce-b8c4-c0c1285a4bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.290117 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.290098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:25.305610 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.305588 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.402968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.402933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.421216 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.421198 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4"] Jul 08 21:14:25.423744 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:25.423715 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode315be0a_8872_40ce_b8c4_c0c1285a4bfa.slice/crio-2677dbe66057ffe09d20af64b9877aaada2436ce0e2447e6db5495753b6ceabb WatchSource:0}: Error finding container 2677dbe66057ffe09d20af64b9877aaada2436ce0e2447e6db5495753b6ceabb: Status 404 returned error can't find the container with id 2677dbe66057ffe09d20af64b9877aaada2436ce0e2447e6db5495753b6ceabb Jul 08 21:14:25.475087 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.475059 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.560321 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.560222 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.661552 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.661519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:25.922849 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:25.922771 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" event={"ID":"e315be0a-8872-40ce-b8c4-c0c1285a4bfa","Type":"ContainerStarted","Data":"2677dbe66057ffe09d20af64b9877aaada2436ce0e2447e6db5495753b6ceabb"} Jul 08 21:14:28.940683 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:28.940654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" event={"ID":"e315be0a-8872-40ce-b8c4-c0c1285a4bfa","Type":"ContainerStarted","Data":"0fb794c9d754c2ad414f733907628396347d31ca0ccfc9d9dd4bbf4510eb6a52"} Jul 08 21:14:29.946831 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:29.946804 2565 generic.go:356] "Generic (PLEG): container finished" podID="e315be0a-8872-40ce-b8c4-c0c1285a4bfa" containerID="0fb794c9d754c2ad414f733907628396347d31ca0ccfc9d9dd4bbf4510eb6a52" exitCode=0 Jul 08 21:14:29.947145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:29.946885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" event={"ID":"e315be0a-8872-40ce-b8c4-c0c1285a4bfa","Type":"ContainerDied","Data":"0fb794c9d754c2ad414f733907628396347d31ca0ccfc9d9dd4bbf4510eb6a52"} Jul 08 21:14:30.954037 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:30.953999 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" event={"ID":"e315be0a-8872-40ce-b8c4-c0c1285a4bfa","Type":"ContainerStarted","Data":"6601fca05a1f5928e91eef45bcec6c3fdcee92af9a7b233d95df8990efaafe46"} Jul 08 21:14:30.954527 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:30.954135 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:30.978225 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:30.978177 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" podStartSLOduration=2.005561968 podStartE2EDuration="6.97816091s" totalImagesPullingTime="3.911383369s" totalInitContainerRuntime="1.061215573s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:30.97542042 +0000 UTC m=+506.547165296" watchObservedRunningTime="2026-07-08 21:14:30.97816091 +0000 UTC m=+506.549905768" Jul 08 21:14:36.817027 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:36.816981 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:41.960080 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:41.960050 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94bc986b9-dcpp4" Jul 08 21:14:44.660385 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:44.660349 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:44.765074 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:44.765040 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:46.580601 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.580569 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfsgx"] Jul 08 21:14:46.587581 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.587562 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.589964 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.589938 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 21:14:46.590089 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.590051 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-v588n\"" Jul 08 21:14:46.591276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.591251 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfsgx"] Jul 08 21:14:46.611740 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.611718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:46.660248 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.660209 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.660489 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.660268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zbrc\" (UniqueName: \"kubernetes.io/projected/f218d1c2-bf24-4766-a4a0-073084d94417-kube-api-access-4zbrc\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.667536 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.667507 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:46.761006 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.760963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.761198 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.761023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zbrc\" (UniqueName: \"kubernetes.io/projected/f218d1c2-bf24-4766-a4a0-073084d94417-kube-api-access-4zbrc\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.761295 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:46.761199 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 21:14:46.761355 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:46.761309 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert podName:f218d1c2-bf24-4766-a4a0-073084d94417 nodeName:}" failed. No retries permitted until 2026-07-08 21:14:47.261285387 +0000 UTC m=+522.833030238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert") pod "odh-model-controller-7998c67dc4-zfsgx" (UID: "f218d1c2-bf24-4766-a4a0-073084d94417") : secret "odh-model-controller-webhook-cert" not found Jul 08 21:14:46.767794 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.767763 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:46.769495 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.769465 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zbrc\" (UniqueName: \"kubernetes.io/projected/f218d1c2-bf24-4766-a4a0-073084d94417-kube-api-access-4zbrc\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:46.919364 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.919279 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:46.977114 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.977086 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-969j8"] Jul 08 21:14:46.979701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.979685 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:46.981929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.981898 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 21:14:46.982047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.981911 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-k6r4z\"" Jul 08 21:14:46.989284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:46.989260 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-969j8"] Jul 08 21:14:47.001451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.001428 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:47.063917 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.063887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hbk\" (UniqueName: \"kubernetes.io/projected/683e2f1e-0255-448c-bbd0-a0e2410f6317-kube-api-access-q4hbk\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.064077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.063926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/683e2f1e-0255-448c-bbd0-a0e2410f6317-tls-certs\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.100813 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.100776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:47.165380 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.165351 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hbk\" (UniqueName: \"kubernetes.io/projected/683e2f1e-0255-448c-bbd0-a0e2410f6317-kube-api-access-q4hbk\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.165566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.165394 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/683e2f1e-0255-448c-bbd0-a0e2410f6317-tls-certs\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.167643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.167614 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/683e2f1e-0255-448c-bbd0-a0e2410f6317-tls-certs\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.173503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.173448 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hbk\" (UniqueName: \"kubernetes.io/projected/683e2f1e-0255-448c-bbd0-a0e2410f6317-kube-api-access-q4hbk\") pod \"model-serving-api-54d89bd5fc-969j8\" (UID: \"683e2f1e-0255-448c-bbd0-a0e2410f6317\") " pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.267362 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.266836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:47.270226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.270175 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f218d1c2-bf24-4766-a4a0-073084d94417-cert\") pod \"odh-model-controller-7998c67dc4-zfsgx\" (UID: \"f218d1c2-bf24-4766-a4a0-073084d94417\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:47.291119 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.291086 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:47.427681 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.427510 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-969j8"] Jul 08 21:14:47.430162 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:47.430134 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683e2f1e_0255_448c_bbd0_a0e2410f6317.slice/crio-15ec21b05d4d79eebc52f1ce47de3d74c84fce8f0fd573b723fe3853b4c8be90 WatchSource:0}: Error finding container 15ec21b05d4d79eebc52f1ce47de3d74c84fce8f0fd573b723fe3853b4c8be90: Status 404 returned error can't find the container with id 15ec21b05d4d79eebc52f1ce47de3d74c84fce8f0fd573b723fe3853b4c8be90 Jul 08 21:14:47.500962 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.500932 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:47.643097 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.643057 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfsgx"] Jul 08 21:14:47.644258 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:47.644211 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf218d1c2_bf24_4766_a4a0_073084d94417.slice/crio-6efdca6dd33ac44887d290465407ad176a14eb66ede545b109d5bebcc7253768 WatchSource:0}: Error finding container 6efdca6dd33ac44887d290465407ad176a14eb66ede545b109d5bebcc7253768: Status 404 returned error can't find the container with id 6efdca6dd33ac44887d290465407ad176a14eb66ede545b109d5bebcc7253768 Jul 08 21:14:47.907062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:47.907015 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:48.031015 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.030973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" event={"ID":"683e2f1e-0255-448c-bbd0-a0e2410f6317","Type":"ContainerStarted","Data":"15ec21b05d4d79eebc52f1ce47de3d74c84fce8f0fd573b723fe3853b4c8be90"} Jul 08 21:14:48.032341 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.032306 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" event={"ID":"f218d1c2-bf24-4766-a4a0-073084d94417","Type":"ContainerStarted","Data":"6efdca6dd33ac44887d290465407ad176a14eb66ede545b109d5bebcc7253768"} Jul 08 21:14:48.062471 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.062439 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt"] Jul 08 21:14:48.066697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.066678 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.068934 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.068910 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 21:14:48.069035 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.068945 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 21:14:48.069994 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.069978 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 21:14:48.070065 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.070039 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:14:48.070112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.070066 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lgtl9\"" Jul 08 21:14:48.076794 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.076767 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt"] Jul 08 21:14:48.175098 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.175015 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.175261 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.175106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db80eb2-0065-455b-bc43-14efbcacf8d9-tmp\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.175261 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.175171 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5w6\" (UniqueName: \"kubernetes.io/projected/8db80eb2-0065-455b-bc43-14efbcacf8d9-kube-api-access-nn5w6\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.276364 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.276332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db80eb2-0065-455b-bc43-14efbcacf8d9-tmp\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.276548 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.276404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5w6\" (UniqueName: \"kubernetes.io/projected/8db80eb2-0065-455b-bc43-14efbcacf8d9-kube-api-access-nn5w6\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.276548 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.276474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.276663 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:48.276618 2565 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 08 21:14:48.276768 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:48.276686 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs podName:8db80eb2-0065-455b-bc43-14efbcacf8d9 nodeName:}" failed. No retries permitted until 2026-07-08 21:14:48.77666091 +0000 UTC m=+524.348405747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs") pod "kube-auth-proxy-5d7c4b48c4-zgwwt" (UID: "8db80eb2-0065-455b-bc43-14efbcacf8d9") : secret "kube-auth-proxy-tls" not found Jul 08 21:14:48.278927 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.278899 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db80eb2-0065-455b-bc43-14efbcacf8d9-tmp\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.284342 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.284319 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5w6\" (UniqueName: \"kubernetes.io/projected/8db80eb2-0065-455b-bc43-14efbcacf8d9-kube-api-access-nn5w6\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.574977 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.574925 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:48.783131 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.783093 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.787847 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.787521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db80eb2-0065-455b-bc43-14efbcacf8d9-tls-certs\") pod \"kube-auth-proxy-5d7c4b48c4-zgwwt\" (UID: \"8db80eb2-0065-455b-bc43-14efbcacf8d9\") " pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.869645 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.869570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:48.875719 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.873855 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:14:48.878003 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.877979 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.882762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.882217 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 21:14:48.882762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.882294 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 21:14:48.882762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.882475 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 21:14:48.882762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.882225 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 21:14:48.882762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.882624 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wvfxn\"" Jul 08 21:14:48.888812 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.888790 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:14:48.927707 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.927674 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:48.980644 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.980618 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" Jul 08 21:14:48.984736 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.984840 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.984840 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984799 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.984918 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.984974 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.984974 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzbn2\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:48.985093 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:48.984987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086231 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.085868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086231 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.085933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzbn2\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086231 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.085965 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086231 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.086005 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.086298 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.086366 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.086560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.086418 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.095968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.086988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.096418 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.096370 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.097032 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.097008 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzbn2\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.102210 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.102185 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.102363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.102258 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.102363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.102289 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.104274 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.104219 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fp5dt\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.140157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.140127 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt"] Jul 08 21:14:49.194302 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.194231 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:49.670636 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:49.670543 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:50.240652 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:50.240614 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db80eb2_0065_455b_bc43_14efbcacf8d9.slice/crio-30b06936aba81be171667dabf5324bb776511d3d1cece774ca29ce9714a63ad4 WatchSource:0}: Error finding container 30b06936aba81be171667dabf5324bb776511d3d1cece774ca29ce9714a63ad4: Status 404 returned error can't find the container with id 30b06936aba81be171667dabf5324bb776511d3d1cece774ca29ce9714a63ad4 Jul 08 21:14:50.395982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:50.395952 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:14:50.949631 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:50.949591 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f40d50c_c4f2_42c9_9fb9_0824ead00b42.slice/crio-0b6b69778324cede2eeeda2b8c50720eafb9276ee55477e1cf3492eea168bd4d WatchSource:0}: Error finding container 0b6b69778324cede2eeeda2b8c50720eafb9276ee55477e1cf3492eea168bd4d: Status 404 returned error can't find the container with id 0b6b69778324cede2eeeda2b8c50720eafb9276ee55477e1cf3492eea168bd4d Jul 08 21:14:51.056632 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:51.056198 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" event={"ID":"0f40d50c-c4f2-42c9-9fb9-0824ead00b42","Type":"ContainerStarted","Data":"0b6b69778324cede2eeeda2b8c50720eafb9276ee55477e1cf3492eea168bd4d"} Jul 08 21:14:51.060677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:51.060646 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" event={"ID":"683e2f1e-0255-448c-bbd0-a0e2410f6317","Type":"ContainerStarted","Data":"dd940373e3afe55d1e35780c954b31a74b19f0b6979879be32bc54522bf8475f"} Jul 08 21:14:51.060788 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:51.060753 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:14:51.062136 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:51.062118 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" event={"ID":"8db80eb2-0065-455b-bc43-14efbcacf8d9","Type":"ContainerStarted","Data":"30b06936aba81be171667dabf5324bb776511d3d1cece774ca29ce9714a63ad4"} Jul 08 21:14:51.075655 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:51.075606 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" podStartSLOduration=2.213865717 podStartE2EDuration="5.075589332s" totalImagesPullingTime="2.861723615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:51.074435354 +0000 UTC m=+526.646180229" watchObservedRunningTime="2026-07-08 21:14:51.075589332 +0000 UTC m=+526.647334192" Jul 08 21:14:52.074037 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:52.073521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" event={"ID":"f218d1c2-bf24-4766-a4a0-073084d94417","Type":"ContainerStarted","Data":"d8c3612f361b52c11c7f74b7a830f06481ffcce0ccdcfc19cb7ee4ffa3ff16c6"} Jul 08 21:14:52.074037 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:52.073987 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:14:52.089013 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:52.088921 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" podStartSLOduration=2.732754823 podStartE2EDuration="6.088900635s" totalImagesPullingTime="3.356145812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:52.088431084 +0000 UTC m=+527.660175959" watchObservedRunningTime="2026-07-08 21:14:52.088900635 +0000 UTC m=+527.660645495" Jul 08 21:14:54.522395 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:54.522347 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:14:54.522683 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:54.522437 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:14:55.096081 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.095995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" event={"ID":"8db80eb2-0065-455b-bc43-14efbcacf8d9","Type":"ContainerStarted","Data":"149afec21b84dc7181bbf1da4fb52c064638ac76bb95d8e6c006033f9fc9640c"} Jul 08 21:14:55.098526 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.098492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" event={"ID":"0f40d50c-c4f2-42c9-9fb9-0824ead00b42","Type":"ContainerStarted","Data":"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123"} Jul 08 21:14:55.099626 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.099604 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:55.101597 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.101568 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fp5dt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 21:14:55.101701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.101618 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 21:14:55.117491 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.117431 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d7c4b48c4-zgwwt" podStartSLOduration=2.836621586 podStartE2EDuration="7.1174109s" totalImagesPullingTime="4.280789314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:55.115676921 +0000 UTC m=+530.687421785" watchObservedRunningTime="2026-07-08 21:14:55.1174109 +0000 UTC m=+530.689155761" Jul 08 21:14:55.147385 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.147323 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" podStartSLOduration=3.577414222 podStartE2EDuration="7.147302529s" totalImagesPullingTime="3.569888307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:14:55.143524647 +0000 UTC m=+530.715269542" watchObservedRunningTime="2026-07-08 21:14:55.147302529 +0000 UTC m=+530.719047390" Jul 08 21:14:55.154319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.154295 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:55.221155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.221117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:55.886120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.886089 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g"] Jul 08 21:14:55.889789 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.889772 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:55.892696 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.892676 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-svt6z\"" Jul 08 21:14:55.919424 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.919405 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g"] Jul 08 21:14:55.945439 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:55.945410 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:56.054923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.054889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.054944 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.054995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055154 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.055286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.055211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbfxg\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-kube-api-access-zbfxg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.104874 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.104846 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:56.115170 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.115140 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:56.155838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.155749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.155838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.155799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.155838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.155825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.155942 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.155989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156047 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156151 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbfxg\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-kube-api-access-zbfxg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156151 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156269 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156269 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156192 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156269 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156208 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156541 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156786 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.156920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.156902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.158365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.158345 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.158479 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.158436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.167312 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.167291 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.167797 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.167760 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbfxg\" (UniqueName: \"kubernetes.io/projected/f8150f62-9c40-4968-b6dc-9f97bc577b1e-kube-api-access-zbfxg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49zf9g\" (UID: \"f8150f62-9c40-4968-b6dc-9f97bc577b1e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.200363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.200340 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:14:56.225940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.225911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:56.364377 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:56.364352 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g"] Jul 08 21:14:56.366091 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:56.366063 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8150f62_9c40_4968_b6dc_9f97bc577b1e.slice/crio-793e387b0497129229de025c8ed3cc068a3702ab8d564915a53365338cd9a2a1 WatchSource:0}: Error finding container 793e387b0497129229de025c8ed3cc068a3702ab8d564915a53365338cd9a2a1: Status 404 returned error can't find the container with id 793e387b0497129229de025c8ed3cc068a3702ab8d564915a53365338cd9a2a1 Jul 08 21:14:57.110692 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.110656 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" event={"ID":"f8150f62-9c40-4968-b6dc-9f97bc577b1e","Type":"ContainerStarted","Data":"793e387b0497129229de025c8ed3cc068a3702ab8d564915a53365338cd9a2a1"} Jul 08 21:14:57.270024 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.269988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:57.309664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.309633 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8"] Jul 08 21:14:57.314391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.314368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:57.317081 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.317057 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cw76b\"" Jul 08 21:14:57.317371 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.317349 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 21:14:57.325571 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.325549 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8"] Jul 08 21:14:57.359988 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.359956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:57.411537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.411455 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:57.466646 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.466619 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj7xm\" (UniqueName: \"kubernetes.io/projected/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-kube-api-access-wj7xm\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:57.466789 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.466680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:57.568013 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.567976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj7xm\" (UniqueName: \"kubernetes.io/projected/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-kube-api-access-wj7xm\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:57.568188 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.568063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:57.568296 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:57.568253 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 21:14:57.568372 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:57.568345 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert podName:7d9c2233-9c8b-4608-89c4-5d0ca59539bd nodeName:}" failed. No retries permitted until 2026-07-08 21:14:58.068323024 +0000 UTC m=+533.640067866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert") pod "kserve-controller-manager-d6f4b65b8-kvzf8" (UID: "7d9c2233-9c8b-4608-89c4-5d0ca59539bd") : secret "kserve-webhook-server-cert" not found Jul 08 21:14:57.586873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:57.586847 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj7xm\" (UniqueName: \"kubernetes.io/projected/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-kube-api-access-wj7xm\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:58.073926 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:58.073890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:58.074113 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:58.074050 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 21:14:58.074168 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:14:58.074115 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert podName:7d9c2233-9c8b-4608-89c4-5d0ca59539bd nodeName:}" failed. No retries permitted until 2026-07-08 21:14:59.07409577 +0000 UTC m=+534.645840608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert") pod "kserve-controller-manager-d6f4b65b8-kvzf8" (UID: "7d9c2233-9c8b-4608-89c4-5d0ca59539bd") : secret "kserve-webhook-server-cert" not found Jul 08 21:14:58.144002 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:58.143971 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:58.567641 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:58.567602 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:58.649920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:58.649891 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:58.750091 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:58.750047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:14:59.083020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.082975 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:59.085697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.085672 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d9c2233-9c8b-4608-89c4-5d0ca59539bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-kvzf8\" (UID: \"7d9c2233-9c8b-4608-89c4-5d0ca59539bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:59.131197 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.131166 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:14:59.223716 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.223682 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:14:59.224012 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.223761 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:14:59.224012 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.223794 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:14:59.274913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.274888 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:14:59.275112 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.275067 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerName="discovery" containerID="cri-o://e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123" gracePeriod=30 Jul 08 21:14:59.296292 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.296268 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8"] Jul 08 21:14:59.297598 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:14:59.297576 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9c2233_9c8b_4608_89c4_5d0ca59539bd.slice/crio-cc54e98b416d202bd78f9e954969b08e2543ccc134d05fca0573e68594c283e5 WatchSource:0}: Error finding container cc54e98b416d202bd78f9e954969b08e2543ccc134d05fca0573e68594c283e5: Status 404 returned error can't find the container with id cc54e98b416d202bd78f9e954969b08e2543ccc134d05fca0573e68594c283e5 Jul 08 21:14:59.524307 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.524284 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:14:59.689036 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.688963 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689036 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.688994 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689036 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689327 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689050 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689327 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689114 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689327 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689154 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzbn2\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689327 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689234 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token\") pod \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\" (UID: \"0f40d50c-c4f2-42c9-9fb9-0824ead00b42\") " Jul 08 21:14:59.689920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.689864 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:14:59.691866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691774 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:14:59.691866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691824 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs" (OuterVolumeSpecName: "local-certs") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:14:59.691866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691856 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:14:59.692020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691865 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2" (OuterVolumeSpecName: "kube-api-access-wzbn2") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "kube-api-access-wzbn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:14:59.692020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691875 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:14:59.692020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.691894 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts" (OuterVolumeSpecName: "cacerts") pod "0f40d50c-c4f2-42c9-9fb9-0824ead00b42" (UID: "0f40d50c-c4f2-42c9-9fb9-0824ead00b42"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:14:59.790501 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790477 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzbn2\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-kube-api-access-wzbn2\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790501 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790499 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-token\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790507 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-cacerts\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790516 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-local-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790524 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-kubeconfig\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790531 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-dns-cert\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:14:59.790630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:14:59.790545 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0f40d50c-c4f2-42c9-9fb9-0824ead00b42-istio-csr-ca-configmap\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:15:00.126557 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.126523 2565 generic.go:356] "Generic (PLEG): container finished" podID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerID="e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123" exitCode=0 Jul 08 21:15:00.126724 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.126566 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" Jul 08 21:15:00.126724 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.126615 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" event={"ID":"0f40d50c-c4f2-42c9-9fb9-0824ead00b42","Type":"ContainerDied","Data":"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123"} Jul 08 21:15:00.126724 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.126667 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt" event={"ID":"0f40d50c-c4f2-42c9-9fb9-0824ead00b42","Type":"ContainerDied","Data":"0b6b69778324cede2eeeda2b8c50720eafb9276ee55477e1cf3492eea168bd4d"} Jul 08 21:15:00.127556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.126693 2565 scope.go:117] "RemoveContainer" containerID="e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123" Jul 08 21:15:00.127839 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.127816 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" event={"ID":"7d9c2233-9c8b-4608-89c4-5d0ca59539bd","Type":"ContainerStarted","Data":"cc54e98b416d202bd78f9e954969b08e2543ccc134d05fca0573e68594c283e5"} Jul 08 21:15:00.129558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.129528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" event={"ID":"f8150f62-9c40-4968-b6dc-9f97bc577b1e","Type":"ContainerStarted","Data":"8471f3b16c6c0717ecc929ac5b4fc56a117f2f948f24ed5ddf3889607f414bf8"} Jul 08 21:15:00.136708 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.136688 2565 scope.go:117] "RemoveContainer" containerID="e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123" Jul 08 21:15:00.136952 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:15:00.136933 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123\": container with ID starting with e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123 not found: ID does not exist" containerID="e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123" Jul 08 21:15:00.137017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.136964 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123"} err="failed to get container status \"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123\": rpc error: code = NotFound desc = could not find container \"e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123\": container with ID starting with e391c77bda0a2835be31d5b60cb711c9f80e242ebfefe9b619246b55925a5123 not found: ID does not exist" Jul 08 21:15:00.150884 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.150841 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" podStartSLOduration=2.295668753 podStartE2EDuration="5.150828089s" totalImagesPullingTime="2.855159336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:00.14932817 +0000 UTC m=+535.721073031" watchObservedRunningTime="2026-07-08 21:15:00.150828089 +0000 UTC m=+535.722572950" Jul 08 21:15:00.162904 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.162880 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:15:00.165327 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.165308 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fp5dt"] Jul 08 21:15:00.200723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.200696 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:15:00.206083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.206062 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:15:00.953419 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:00.951519 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" path="/var/lib/kubelet/pods/0f40d50c-c4f2-42c9-9fb9-0824ead00b42/volumes" Jul 08 21:15:01.135572 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:01.135478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:15:01.136512 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:01.136486 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49zf9g" Jul 08 21:15:02.078084 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:02.078055 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-969j8" Jul 08 21:15:03.148826 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:03.148794 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" event={"ID":"7d9c2233-9c8b-4608-89c4-5d0ca59539bd","Type":"ContainerStarted","Data":"c8377cc9c3fa73ec7e704107a08067ea92dcae459ea1148ad5f0da0e016c97b4"} Jul 08 21:15:03.149188 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:03.148959 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:15:03.164827 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:03.164778 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" podStartSLOduration=2.486659596 podStartE2EDuration="6.164763989s" totalImagesPullingTime="3.678104393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:03.163110665 +0000 UTC m=+538.734855525" watchObservedRunningTime="2026-07-08 21:15:03.164763989 +0000 UTC m=+538.736508848" Jul 08 21:15:04.088499 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:04.088473 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zfsgx" Jul 08 21:15:05.421803 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:05.421774 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:05.473315 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:05.473287 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:34.162460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:34.162432 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kvzf8" Jul 08 21:15:46.317866 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.317837 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:46.318363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.318261 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerName="discovery" Jul 08 21:15:46.318363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.318274 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerName="discovery" Jul 08 21:15:46.318443 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.318403 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f40d50c-c4f2-42c9-9fb9-0824ead00b42" containerName="discovery" Jul 08 21:15:46.323562 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.323542 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:46.326217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.326195 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:15:46.328002 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.327982 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:15:46.328227 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.328208 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cq5wp\"" Jul 08 21:15:46.338962 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.338942 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:46.380285 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.380250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:46.456405 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.456384 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxrws\" (UniqueName: \"kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws\") pod \"kuadrant-operator-catalog-6xshz\" (UID: \"aaaeb708-5961-45e6-9d76-58d59b5fae8b\") " pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:46.503055 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.503019 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:46.556850 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.556821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxrws\" (UniqueName: \"kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws\") pod \"kuadrant-operator-catalog-6xshz\" (UID: \"aaaeb708-5961-45e6-9d76-58d59b5fae8b\") " pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:46.564643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.564622 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxrws\" (UniqueName: \"kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws\") pod \"kuadrant-operator-catalog-6xshz\" (UID: \"aaaeb708-5961-45e6-9d76-58d59b5fae8b\") " pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:46.633895 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.633839 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:46.760134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.760111 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:46.762252 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:15:46.762206 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaaeb708_5961_45e6_9d76_58d59b5fae8b.slice/crio-61709e39d42d88dae6848d26a3c0f42e5a958cb35dc5da0787fc31916604db39 WatchSource:0}: Error finding container 61709e39d42d88dae6848d26a3c0f42e5a958cb35dc5da0787fc31916604db39: Status 404 returned error can't find the container with id 61709e39d42d88dae6848d26a3c0f42e5a958cb35dc5da0787fc31916604db39 Jul 08 21:15:46.788663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.788643 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:46.897621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.897569 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7cqx"] Jul 08 21:15:46.900562 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.900547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:46.907646 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.907625 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7cqx"] Jul 08 21:15:46.960753 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:46.960731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml77j\" (UniqueName: \"kubernetes.io/projected/601d6727-d641-4d2f-8dd9-525dec415fd3-kube-api-access-ml77j\") pod \"kuadrant-operator-catalog-k7cqx\" (UID: \"601d6727-d641-4d2f-8dd9-525dec415fd3\") " pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:47.061472 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.061448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml77j\" (UniqueName: \"kubernetes.io/projected/601d6727-d641-4d2f-8dd9-525dec415fd3-kube-api-access-ml77j\") pod \"kuadrant-operator-catalog-k7cqx\" (UID: \"601d6727-d641-4d2f-8dd9-525dec415fd3\") " pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:47.069594 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.069568 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml77j\" (UniqueName: \"kubernetes.io/projected/601d6727-d641-4d2f-8dd9-525dec415fd3-kube-api-access-ml77j\") pod \"kuadrant-operator-catalog-k7cqx\" (UID: \"601d6727-d641-4d2f-8dd9-525dec415fd3\") " pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:47.211207 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.211130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:47.318441 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.318375 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:47.359921 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.359854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" event={"ID":"aaaeb708-5961-45e6-9d76-58d59b5fae8b","Type":"ContainerStarted","Data":"61709e39d42d88dae6848d26a3c0f42e5a958cb35dc5da0787fc31916604db39"} Jul 08 21:15:47.361802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.361778 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7cqx"] Jul 08 21:15:47.368885 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:15:47.368862 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601d6727_d641_4d2f_8dd9_525dec415fd3.slice/crio-efb33f403f248061b651a8052d22f388e12e8c36e07bdbfbce8c9da9f98910d7 WatchSource:0}: Error finding container efb33f403f248061b651a8052d22f388e12e8c36e07bdbfbce8c9da9f98910d7: Status 404 returned error can't find the container with id efb33f403f248061b651a8052d22f388e12e8c36e07bdbfbce8c9da9f98910d7 Jul 08 21:15:47.372154 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:47.372107 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:48.367729 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:48.367698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" event={"ID":"601d6727-d641-4d2f-8dd9-525dec415fd3","Type":"ContainerStarted","Data":"efb33f403f248061b651a8052d22f388e12e8c36e07bdbfbce8c9da9f98910d7"} Jul 08 21:15:49.379439 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.379410 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" event={"ID":"aaaeb708-5961-45e6-9d76-58d59b5fae8b","Type":"ContainerStarted","Data":"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7"} Jul 08 21:15:49.379844 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.379463 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" podUID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" containerName="registry-server" containerID="cri-o://f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7" gracePeriod=2 Jul 08 21:15:49.381278 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.381256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" event={"ID":"601d6727-d641-4d2f-8dd9-525dec415fd3","Type":"ContainerStarted","Data":"9680bf1fc1add34441057ca7d4605cc77b160dfd60812a446d185fc50e1b95f1"} Jul 08 21:15:49.393527 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.393486 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" podStartSLOduration=0.870933163 podStartE2EDuration="3.393472653s" totalImagesPullingTime="2.52253949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:49.393224522 +0000 UTC m=+584.964969386" watchObservedRunningTime="2026-07-08 21:15:49.393472653 +0000 UTC m=+584.965217508" Jul 08 21:15:49.407339 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.407294 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" podStartSLOduration=1.4951997320000001 podStartE2EDuration="3.407279343s" totalImagesPullingTime="1.912079611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:49.406100398 +0000 UTC m=+584.977845258" watchObservedRunningTime="2026-07-08 21:15:49.407279343 +0000 UTC m=+584.979024201" Jul 08 21:15:49.616553 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.616532 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xshz_aaaeb708-5961-45e6-9d76-58d59b5fae8b/registry-server/0.log" Jul 08 21:15:49.618783 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.618764 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:49.686884 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.686859 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxrws\" (UniqueName: \"kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws\") pod \"aaaeb708-5961-45e6-9d76-58d59b5fae8b\" (UID: \"aaaeb708-5961-45e6-9d76-58d59b5fae8b\") " Jul 08 21:15:49.688864 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.688839 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws" (OuterVolumeSpecName: "kube-api-access-jxrws") pod "aaaeb708-5961-45e6-9d76-58d59b5fae8b" (UID: "aaaeb708-5961-45e6-9d76-58d59b5fae8b"). InnerVolumeSpecName "kube-api-access-jxrws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:15:49.787806 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:49.787779 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxrws\" (UniqueName: \"kubernetes.io/projected/aaaeb708-5961-45e6-9d76-58d59b5fae8b-kube-api-access-jxrws\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:15:50.387556 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387531 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xshz_aaaeb708-5961-45e6-9d76-58d59b5fae8b/registry-server/0.log" Jul 08 21:15:50.387945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387572 2565 generic.go:356] "Generic (PLEG): container finished" podID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" containerID="f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7" exitCode=2 Jul 08 21:15:50.387945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387615 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" Jul 08 21:15:50.387945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" event={"ID":"aaaeb708-5961-45e6-9d76-58d59b5fae8b","Type":"ContainerDied","Data":"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7"} Jul 08 21:15:50.387945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xshz" event={"ID":"aaaeb708-5961-45e6-9d76-58d59b5fae8b","Type":"ContainerDied","Data":"61709e39d42d88dae6848d26a3c0f42e5a958cb35dc5da0787fc31916604db39"} Jul 08 21:15:50.387945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.387716 2565 scope.go:117] "RemoveContainer" containerID="f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7" Jul 08 21:15:50.396126 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.396111 2565 scope.go:117] "RemoveContainer" containerID="f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7" Jul 08 21:15:50.396408 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:15:50.396389 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7\": container with ID starting with f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7 not found: ID does not exist" containerID="f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7" Jul 08 21:15:50.396467 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.396417 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7"} err="failed to get container status \"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7\": rpc error: code = NotFound desc = could not find container \"f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7\": container with ID starting with f79d6fcf6f5769e878b92018af291593eb95c96851f766ee2910f0013734a9c7 not found: ID does not exist" Jul 08 21:15:50.408661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.408637 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:50.412262 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.412230 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xshz"] Jul 08 21:15:50.949479 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:50.949454 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" path="/var/lib/kubelet/pods/aaaeb708-5961-45e6-9d76-58d59b5fae8b/volumes" Jul 08 21:15:56.417978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:56.417929 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:56.519635 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:56.519601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:57.212122 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:57.212096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:57.212299 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:57.212149 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:57.233522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:57.233493 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:57.447427 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:57.447404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k7cqx" Jul 08 21:15:58.355433 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:58.355390 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:15:58.456288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:15:58.456258 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:04.866466 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:04.866437 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:16:04.868373 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:04.868352 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:16:05.476822 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.476788 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c66f5f9f-fsqz8"] Jul 08 21:16:05.477190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.477177 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" containerName="registry-server" Jul 08 21:16:05.477258 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.477192 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" containerName="registry-server" Jul 08 21:16:05.477315 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.477304 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaaeb708-5961-45e6-9d76-58d59b5fae8b" containerName="registry-server" Jul 08 21:16:05.481658 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.481637 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.485132 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485108 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 21:16:05.485293 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485107 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r54dz\"" Jul 08 21:16:05.485293 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485108 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 21:16:05.485293 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485107 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 21:16:05.485445 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485174 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 21:16:05.485445 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.485266 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 21:16:05.489900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.489879 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 21:16:05.492672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.492651 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c66f5f9f-fsqz8"] Jul 08 21:16:05.526860 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.526835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:05.612501 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-console-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.612501 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-oauth-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-service-ca\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612523 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-trusted-ca-bundle\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612618 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-oauth-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.612666 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccqq\" (UniqueName: \"kubernetes.io/projected/160c488d-b811-493f-b80b-3cc45dd79e12-kube-api-access-6ccqq\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.635885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.626269 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:05.713468 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-console-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713582 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-oauth-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713582 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-service-ca\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713582 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-trusted-ca-bundle\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713592 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-oauth-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.713748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.713688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccqq\" (UniqueName: \"kubernetes.io/projected/160c488d-b811-493f-b80b-3cc45dd79e12-kube-api-access-6ccqq\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.714272 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.714226 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-console-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.714373 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.714276 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-service-ca\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.714437 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.714405 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-trusted-ca-bundle\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.714499 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.714456 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/160c488d-b811-493f-b80b-3cc45dd79e12-oauth-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.716083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.716061 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-oauth-config\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.716190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.716176 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/160c488d-b811-493f-b80b-3cc45dd79e12-console-serving-cert\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.720744 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.720721 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccqq\" (UniqueName: \"kubernetes.io/projected/160c488d-b811-493f-b80b-3cc45dd79e12-kube-api-access-6ccqq\") pod \"console-77c66f5f9f-fsqz8\" (UID: \"160c488d-b811-493f-b80b-3cc45dd79e12\") " pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.794509 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.794491 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:05.921256 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:05.921133 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c66f5f9f-fsqz8"] Jul 08 21:16:05.923830 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:05.923801 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160c488d_b811_493f_b80b_3cc45dd79e12.slice/crio-d2ea9b282d97c15462f62ec7f87adca7bc69aad5c12700ccb69f5539d3341323 WatchSource:0}: Error finding container d2ea9b282d97c15462f62ec7f87adca7bc69aad5c12700ccb69f5539d3341323: Status 404 returned error can't find the container with id d2ea9b282d97c15462f62ec7f87adca7bc69aad5c12700ccb69f5539d3341323 Jul 08 21:16:06.467578 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:06.467545 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c66f5f9f-fsqz8" event={"ID":"160c488d-b811-493f-b80b-3cc45dd79e12","Type":"ContainerStarted","Data":"2f9793bad722db47ef53a7a85fea2bebb441cce5501ae1821ce48da6e0e113ed"} Jul 08 21:16:06.467764 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:06.467601 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c66f5f9f-fsqz8" event={"ID":"160c488d-b811-493f-b80b-3cc45dd79e12","Type":"ContainerStarted","Data":"d2ea9b282d97c15462f62ec7f87adca7bc69aad5c12700ccb69f5539d3341323"} Jul 08 21:16:06.482929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:06.482886 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c66f5f9f-fsqz8" podStartSLOduration=1.482873057 podStartE2EDuration="1.482873057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:06.482130546 +0000 UTC m=+602.053875407" watchObservedRunningTime="2026-07-08 21:16:06.482873057 +0000 UTC m=+602.054617915" Jul 08 21:16:15.595322 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:15.595285 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:15.794857 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:15.794829 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:15.795006 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:15.794878 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:15.799634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:15.799613 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:16.520356 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:16.520328 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c66f5f9f-fsqz8" Jul 08 21:16:17.461399 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:17.461370 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:18.259120 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.259089 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd"] Jul 08 21:16:18.266014 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.265995 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:18.268404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.268383 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2nbkh\"" Jul 08 21:16:18.271592 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.271573 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd"] Jul 08 21:16:18.308588 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.308562 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgpkn\" (UniqueName: \"kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn\") pod \"limitador-operator-controller-manager-859d7b78bd-28dgd\" (UID: \"d92d954f-4085-40b5-80e4-cd9d94af375d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:18.409331 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.409309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgpkn\" (UniqueName: \"kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn\") pod \"limitador-operator-controller-manager-859d7b78bd-28dgd\" (UID: \"d92d954f-4085-40b5-80e4-cd9d94af375d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:18.420152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.420128 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgpkn\" (UniqueName: \"kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn\") pod \"limitador-operator-controller-manager-859d7b78bd-28dgd\" (UID: \"d92d954f-4085-40b5-80e4-cd9d94af375d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:18.577956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.577933 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:18.706017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:18.705995 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd"] Jul 08 21:16:18.708226 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:18.708201 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92d954f_4085_40b5_80e4_cd9d94af375d.slice/crio-9cffb4dac2659644f8be241e544338590953e6be27449e54e74d1f38c520dfce WatchSource:0}: Error finding container 9cffb4dac2659644f8be241e544338590953e6be27449e54e74d1f38c520dfce: Status 404 returned error can't find the container with id 9cffb4dac2659644f8be241e544338590953e6be27449e54e74d1f38c520dfce Jul 08 21:16:19.535488 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:19.535453 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" event={"ID":"d92d954f-4085-40b5-80e4-cd9d94af375d","Type":"ContainerStarted","Data":"9cffb4dac2659644f8be241e544338590953e6be27449e54e74d1f38c520dfce"} Jul 08 21:16:19.768874 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:19.768839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:19.870505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:19.870411 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:21.547097 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:21.547064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" event={"ID":"d92d954f-4085-40b5-80e4-cd9d94af375d","Type":"ContainerStarted","Data":"ef5dad525ed6c96f44f2dbc95c81d9412613546e92ad5093a20fd188be10c3d5"} Jul 08 21:16:21.547551 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:21.547199 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:21.562397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:21.562339 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" podStartSLOduration=1.525481259 podStartE2EDuration="3.562319141s" totalImagesPullingTime="2.036837882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:21.561380857 +0000 UTC m=+617.133125716" watchObservedRunningTime="2026-07-08 21:16:21.562319141 +0000 UTC m=+617.134064000" Jul 08 21:16:22.308580 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.308554 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khvcx"] Jul 08 21:16:22.311965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.311945 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:22.314276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.314251 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-64t7f\"" Jul 08 21:16:22.328170 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.328150 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khvcx"] Jul 08 21:16:22.345658 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.345629 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlm2w\" (UniqueName: \"kubernetes.io/projected/1ef84a54-0db0-4d7b-969e-6e865da7350d-kube-api-access-mlm2w\") pod \"authorino-operator-75cc8d5cb7-khvcx\" (UID: \"1ef84a54-0db0-4d7b-969e-6e865da7350d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:22.446497 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.446466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlm2w\" (UniqueName: \"kubernetes.io/projected/1ef84a54-0db0-4d7b-969e-6e865da7350d-kube-api-access-mlm2w\") pod \"authorino-operator-75cc8d5cb7-khvcx\" (UID: \"1ef84a54-0db0-4d7b-969e-6e865da7350d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:22.463452 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.463428 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlm2w\" (UniqueName: \"kubernetes.io/projected/1ef84a54-0db0-4d7b-969e-6e865da7350d-kube-api-access-mlm2w\") pod \"authorino-operator-75cc8d5cb7-khvcx\" (UID: \"1ef84a54-0db0-4d7b-969e-6e865da7350d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:22.621595 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.621525 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:22.759349 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:22.759322 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khvcx"] Jul 08 21:16:22.761560 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:22.761534 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef84a54_0db0_4d7b_969e_6e865da7350d.slice/crio-4ab82fdbab1155d0d38116dd2dcda69c937ffd63d1bdd91c8b5a4098ecea9ff4 WatchSource:0}: Error finding container 4ab82fdbab1155d0d38116dd2dcda69c937ffd63d1bdd91c8b5a4098ecea9ff4: Status 404 returned error can't find the container with id 4ab82fdbab1155d0d38116dd2dcda69c937ffd63d1bdd91c8b5a4098ecea9ff4 Jul 08 21:16:23.037457 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:23.037426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:23.142828 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:23.142788 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:23.411437 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:23.411316 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:23.557641 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:23.557607 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" event={"ID":"1ef84a54-0db0-4d7b-969e-6e865da7350d","Type":"ContainerStarted","Data":"4ab82fdbab1155d0d38116dd2dcda69c937ffd63d1bdd91c8b5a4098ecea9ff4"} Jul 08 21:16:25.028260 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.028196 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:25.570004 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.569969 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" event={"ID":"1ef84a54-0db0-4d7b-969e-6e865da7350d","Type":"ContainerStarted","Data":"89c5b819b63f9df276f854f34067b545db46a074b9896988058d5c78b3634471"} Jul 08 21:16:25.570168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.570132 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:25.589147 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.589102 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" podStartSLOduration=0.96393249 podStartE2EDuration="3.589086042s" totalImagesPullingTime="2.625153552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:25.587928253 +0000 UTC m=+621.159673107" watchObservedRunningTime="2026-07-08 21:16:25.589086042 +0000 UTC m=+621.160830905" Jul 08 21:16:25.634182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.634159 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96"] Jul 08 21:16:25.637878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.637826 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:25.641046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.641027 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vfhbd\"" Jul 08 21:16:25.641271 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.641224 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 21:16:25.651669 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.651652 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96"] Jul 08 21:16:25.673778 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.673752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2tx\" (UniqueName: \"kubernetes.io/projected/4646a204-e883-4e2b-ba75-d3f45949af17-kube-api-access-6b2tx\") pod \"dns-operator-controller-manager-864dd78c4c-99n96\" (UID: \"4646a204-e883-4e2b-ba75-d3f45949af17\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:25.774731 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.774706 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2tx\" (UniqueName: \"kubernetes.io/projected/4646a204-e883-4e2b-ba75-d3f45949af17-kube-api-access-6b2tx\") pod \"dns-operator-controller-manager-864dd78c4c-99n96\" (UID: \"4646a204-e883-4e2b-ba75-d3f45949af17\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:25.782463 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.782443 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2tx\" (UniqueName: \"kubernetes.io/projected/4646a204-e883-4e2b-ba75-d3f45949af17-kube-api-access-6b2tx\") pod \"dns-operator-controller-manager-864dd78c4c-99n96\" (UID: \"4646a204-e883-4e2b-ba75-d3f45949af17\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:25.948588 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:25.948538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:26.089602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:26.089576 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96"] Jul 08 21:16:26.090752 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:26.090726 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4646a204_e883_4e2b_ba75_d3f45949af17.slice/crio-86a6fc1b740d782fa591957d91c8bdbc8d71a28708f5ab98465801b24f27a9d0 WatchSource:0}: Error finding container 86a6fc1b740d782fa591957d91c8bdbc8d71a28708f5ab98465801b24f27a9d0: Status 404 returned error can't find the container with id 86a6fc1b740d782fa591957d91c8bdbc8d71a28708f5ab98465801b24f27a9d0 Jul 08 21:16:26.212035 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:26.211944 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:26.321626 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:26.321580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:26.581184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:26.581144 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" event={"ID":"4646a204-e883-4e2b-ba75-d3f45949af17","Type":"ContainerStarted","Data":"86a6fc1b740d782fa591957d91c8bdbc8d71a28708f5ab98465801b24f27a9d0"} Jul 08 21:16:28.031607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:28.031545 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:29.602152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:29.602118 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" event={"ID":"4646a204-e883-4e2b-ba75-d3f45949af17","Type":"ContainerStarted","Data":"0eb1d2e270cd0ef17350a253c6520923d1ecd6b6e6bb0c82883feeb8ec529ddd"} Jul 08 21:16:29.602545 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:29.602283 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:29.617354 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:29.617316 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" podStartSLOduration=1.5605899600000002 podStartE2EDuration="4.6173027s" totalImagesPullingTime="3.05671274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:29.616106275 +0000 UTC m=+625.187851135" watchObservedRunningTime="2026-07-08 21:16:29.6173027 +0000 UTC m=+625.189047558" Jul 08 21:16:30.038440 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.038414 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445"] Jul 08 21:16:30.041955 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.041935 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.044473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.044454 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wkt97\"" Jul 08 21:16:30.069315 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.069288 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445"] Jul 08 21:16:30.114115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.114086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.114265 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.114127 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xrqz\" (UniqueName: \"kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.215549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.215524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.215662 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.215552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xrqz\" (UniqueName: \"kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.215951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.215934 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.223062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.223037 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xrqz\" (UniqueName: \"kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.352086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.352026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:30.505437 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.505411 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445"] Jul 08 21:16:30.506415 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:30.506387 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a2d446_bbcd_400b_9b1c_64c8428dc004.slice/crio-59818f2e5cb2b7c70df3f9835b06bce3ee4ce999760d181ecd923d30ed0c5cd4 WatchSource:0}: Error finding container 59818f2e5cb2b7c70df3f9835b06bce3ee4ce999760d181ecd923d30ed0c5cd4: Status 404 returned error can't find the container with id 59818f2e5cb2b7c70df3f9835b06bce3ee4ce999760d181ecd923d30ed0c5cd4 Jul 08 21:16:30.608528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:30.608464 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" event={"ID":"24a2d446-bbcd-400b-9b1c-64c8428dc004","Type":"ContainerStarted","Data":"59818f2e5cb2b7c70df3f9835b06bce3ee4ce999760d181ecd923d30ed0c5cd4"} Jul 08 21:16:31.219209 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:31.219184 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:31.319902 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:31.319873 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:32.553537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:32.553509 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:33.822211 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:33.822174 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:34.432596 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:34.432564 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:36.583059 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:36.583032 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khvcx" Jul 08 21:16:36.609626 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:36.609600 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:36.666617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:36.666591 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:37.034017 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:37.033953 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:37.127583 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:37.127542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:38.653139 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:38.653109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" event={"ID":"24a2d446-bbcd-400b-9b1c-64c8428dc004","Type":"ContainerStarted","Data":"6dca9cf303619c674546e5546233b633c39f0258f5e1f58ccdd511b10e0126dc"} Jul 08 21:16:38.653554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:38.653185 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:38.671399 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:38.671353 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" podStartSLOduration=0.999560991 podStartE2EDuration="8.671337008s" totalImagesPullingTime="7.671776017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:38.670159469 +0000 UTC m=+634.241904327" watchObservedRunningTime="2026-07-08 21:16:38.671337008 +0000 UTC m=+634.243081867" Jul 08 21:16:40.002050 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.002009 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:40.102346 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.102292 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:40.611074 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.611042 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-99n96" Jul 08 21:16:40.810335 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.810294 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x"] Jul 08 21:16:40.814350 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.814323 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:40.816574 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.816555 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 21:16:40.816574 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.816567 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 21:16:40.816754 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.816738 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dfkr2\"" Jul 08 21:16:40.820940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.820918 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x"] Jul 08 21:16:40.877402 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.877324 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:40.907463 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.907438 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e06d0a8-7e51-4db1-87f6-078aaf947050-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:40.907585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.907475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e06d0a8-7e51-4db1-87f6-078aaf947050-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:40.907628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:40.907578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dqzv\" (UniqueName: \"kubernetes.io/projected/2e06d0a8-7e51-4db1-87f6-078aaf947050-kube-api-access-6dqzv\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.008543 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.008510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e06d0a8-7e51-4db1-87f6-078aaf947050-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.008920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.008553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e06d0a8-7e51-4db1-87f6-078aaf947050-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.008920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.008610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dqzv\" (UniqueName: \"kubernetes.io/projected/2e06d0a8-7e51-4db1-87f6-078aaf947050-kube-api-access-6dqzv\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.009306 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.009285 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e06d0a8-7e51-4db1-87f6-078aaf947050-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.010971 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.010942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e06d0a8-7e51-4db1-87f6-078aaf947050-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.028863 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.028841 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dqzv\" (UniqueName: \"kubernetes.io/projected/2e06d0a8-7e51-4db1-87f6-078aaf947050-kube-api-access-6dqzv\") pod \"kuadrant-console-plugin-66b5774985-d7w5x\" (UID: \"2e06d0a8-7e51-4db1-87f6-078aaf947050\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.048486 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.048458 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:41.124313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.124290 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" Jul 08 21:16:41.261694 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.261671 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x"] Jul 08 21:16:41.263999 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:41.263974 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e06d0a8_7e51_4db1_87f6_078aaf947050.slice/crio-86985b184bc6ca6998c6468e1c2b31ed6dafbd898361a3608c9203f45fbfb26d WatchSource:0}: Error finding container 86985b184bc6ca6998c6468e1c2b31ed6dafbd898361a3608c9203f45fbfb26d: Status 404 returned error can't find the container with id 86985b184bc6ca6998c6468e1c2b31ed6dafbd898361a3608c9203f45fbfb26d Jul 08 21:16:41.671366 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:41.671337 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" event={"ID":"2e06d0a8-7e51-4db1-87f6-078aaf947050","Type":"ContainerStarted","Data":"86985b184bc6ca6998c6468e1c2b31ed6dafbd898361a3608c9203f45fbfb26d"} Jul 08 21:16:44.166529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:44.166492 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:44.225522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:44.225472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:49.660617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:49.660584 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:50.486692 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.486660 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj"] Jul 08 21:16:50.490631 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.490610 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.502546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.502526 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj"] Jul 08 21:16:50.596005 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.595967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfk4l\" (UniqueName: \"kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.596175 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.596107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.696812 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.696778 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfk4l\" (UniqueName: \"kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.697150 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.696861 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.697191 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.697168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.705009 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.704990 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfk4l\" (UniqueName: \"kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.801544 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.801512 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:50.853920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.853872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:50.956219 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.956199 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj"] Jul 08 21:16:50.957593 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:50.957567 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:50.959373 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:16:50.959347 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2e283d_f7f2_47f3_9934_4b9ae3818bb6.slice/crio-2d8e5339e60125bc9997020aa4de0fd27c6ab0fa245a002b3a06ef50e35eef25 WatchSource:0}: Error finding container 2d8e5339e60125bc9997020aa4de0fd27c6ab0fa245a002b3a06ef50e35eef25: Status 404 returned error can't find the container with id 2d8e5339e60125bc9997020aa4de0fd27c6ab0fa245a002b3a06ef50e35eef25 Jul 08 21:16:51.352452 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.352415 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj"] Jul 08 21:16:51.357878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.357854 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj"] Jul 08 21:16:51.368799 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.368779 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445"] Jul 08 21:16:51.368965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.368940 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" containerName="manager" containerID="cri-o://6dca9cf303619c674546e5546233b633c39f0258f5e1f58ccdd511b10e0126dc" gracePeriod=2 Jul 08 21:16:51.373566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.373542 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:16:51.374401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.374382 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerName="manager" Jul 08 21:16:51.374401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.374403 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerName="manager" Jul 08 21:16:51.374541 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.374530 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerName="manager" Jul 08 21:16:51.379427 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.379062 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.389826 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.389804 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445"] Jul 08 21:16:51.391381 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.391342 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"24a2d446-bbcd-400b-9b1c-64c8428dc004\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-qw445\": pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:51.393768 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.393734 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:51.394871 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.393821 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:16:51.396926 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.396909 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:16:51.397291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.397276 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" containerName="manager" Jul 08 21:16:51.397291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.397291 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" containerName="manager" Jul 08 21:16:51.397379 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.397370 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" containerName="manager" Jul 08 21:16:51.399926 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.399906 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.400094 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.400077 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd"] Jul 08 21:16:51.400265 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.400217 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerName="manager" containerID="cri-o://ef5dad525ed6c96f44f2dbc95c81d9412613546e92ad5093a20fd188be10c3d5" gracePeriod=2 Jul 08 21:16:51.408647 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.408626 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd"] Jul 08 21:16:51.411189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.411163 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:16:51.419057 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.419030 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:51.420971 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.420949 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:51.423174 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.423151 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj"] Jul 08 21:16:51.423549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.423536 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerName="manager" Jul 08 21:16:51.423589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.423550 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerName="manager" Jul 08 21:16:51.423668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.423635 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerName="manager" Jul 08 21:16:51.426758 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.426737 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:16:51.439077 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.439051 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj"] Jul 08 21:16:51.465166 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.465055 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:51.467076 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.467047 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:51.504924 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.504902 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sttg\" (UniqueName: \"kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.505045 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.504946 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp7hr\" (UniqueName: \"kubernetes.io/projected/f04e6e8b-8e33-4c52-89a5-ceadbe32996b-kube-api-access-qp7hr\") pod \"limitador-operator-controller-manager-859d7b78bd-nxghj\" (UID: \"f04e6e8b-8e33-4c52-89a5-ceadbe32996b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:16:51.505045 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.504976 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.505373 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.505052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbslq\" (UniqueName: \"kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.505373 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.505089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.605946 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.605873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbslq\" (UniqueName: \"kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.605946 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.605931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.606141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.606043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sttg\" (UniqueName: \"kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.606141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.606076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp7hr\" (UniqueName: \"kubernetes.io/projected/f04e6e8b-8e33-4c52-89a5-ceadbe32996b-kube-api-access-qp7hr\") pod \"limitador-operator-controller-manager-859d7b78bd-nxghj\" (UID: \"f04e6e8b-8e33-4c52-89a5-ceadbe32996b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:16:51.606141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.606103 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.606428 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.606404 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.606553 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.606533 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.615539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.615509 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbslq\" (UniqueName: \"kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mb6gf\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.615951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.615924 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sttg\" (UniqueName: \"kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xmqm4\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.616755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.616730 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp7hr\" (UniqueName: \"kubernetes.io/projected/f04e6e8b-8e33-4c52-89a5-ceadbe32996b-kube-api-access-qp7hr\") pod \"limitador-operator-controller-manager-859d7b78bd-nxghj\" (UID: \"f04e6e8b-8e33-4c52-89a5-ceadbe32996b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:16:51.747855 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.747654 2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" container="manager" Jul 08 21:16:51.750025 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.749969 2565 generic.go:356] "Generic (PLEG): container finished" podID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerID="ef5dad525ed6c96f44f2dbc95c81d9412613546e92ad5093a20fd188be10c3d5" exitCode=0 Jul 08 21:16:51.750222 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.750195 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:16:51.751891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.751866 2565 generic.go:356] "Generic (PLEG): container finished" podID="24a2d446-bbcd-400b-9b1c-64c8428dc004" containerID="6dca9cf303619c674546e5546233b633c39f0258f5e1f58ccdd511b10e0126dc" exitCode=0 Jul 08 21:16:51.752211 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.752183 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:16:51.754279 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:16:51.754228 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:16:51.789183 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.789165 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:16:51.796566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.796546 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:16:51.802459 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:51.802442 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:16:53.552073 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:53.552015 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.40:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 21:16:54.638874 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:54.638839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:54.741073 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:54.741039 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:54.835657 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:54.835601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:58.361703 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:58.361670 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:16:58.412835 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:16:58.412796 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:02.898089 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:02.898046 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:02.998440 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:02.998398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:05.464391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.464319 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.466612 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.466580 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.477179 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.477080 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.479670 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.479640 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.481476 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.481450 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.549793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.549570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume\") pod \"24a2d446-bbcd-400b-9b1c-64c8428dc004\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " Jul 08 21:17:05.549793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.549622 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xrqz\" (UniqueName: \"kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz\") pod \"24a2d446-bbcd-400b-9b1c-64c8428dc004\" (UID: \"24a2d446-bbcd-400b-9b1c-64c8428dc004\") " Jul 08 21:17:05.549793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.549703 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgpkn\" (UniqueName: \"kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn\") pod \"d92d954f-4085-40b5-80e4-cd9d94af375d\" (UID: \"d92d954f-4085-40b5-80e4-cd9d94af375d\") " Jul 08 21:17:05.551395 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.550625 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24a2d446-bbcd-400b-9b1c-64c8428dc004" (UID: "24a2d446-bbcd-400b-9b1c-64c8428dc004"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:17:05.553418 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.553292 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz" (OuterVolumeSpecName: "kube-api-access-5xrqz") pod "24a2d446-bbcd-400b-9b1c-64c8428dc004" (UID: "24a2d446-bbcd-400b-9b1c-64c8428dc004"). InnerVolumeSpecName "kube-api-access-5xrqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:17:05.554873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.554829 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn" (OuterVolumeSpecName: "kube-api-access-kgpkn") pod "d92d954f-4085-40b5-80e4-cd9d94af375d" (UID: "d92d954f-4085-40b5-80e4-cd9d94af375d"). InnerVolumeSpecName "kube-api-access-kgpkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:17:05.633571 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.633544 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj"] Jul 08 21:17:05.634986 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:05.634959 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04e6e8b_8e33_4c52_89a5_ceadbe32996b.slice/crio-7aaab41621996232c922fa8de6779c734c2d458fc361c7aecffe8237498d6df2 WatchSource:0}: Error finding container 7aaab41621996232c922fa8de6779c734c2d458fc361c7aecffe8237498d6df2: Status 404 returned error can't find the container with id 7aaab41621996232c922fa8de6779c734c2d458fc361c7aecffe8237498d6df2 Jul 08 21:17:05.652913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.651438 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a2d446-bbcd-400b-9b1c-64c8428dc004-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:05.652913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.651468 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xrqz\" (UniqueName: \"kubernetes.io/projected/24a2d446-bbcd-400b-9b1c-64c8428dc004-kube-api-access-5xrqz\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:05.652913 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.651486 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgpkn\" (UniqueName: \"kubernetes.io/projected/d92d954f-4085-40b5-80e4-cd9d94af375d-kube-api-access-kgpkn\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:05.806848 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.806817 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:17:05.809897 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:05.809869 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06251dc_8496_4ac1_8197_78be226c9788.slice/crio-d48f679e0e45e46e651dc7313b4bd431d22e0145811249e11df96672e572e71c WatchSource:0}: Error finding container d48f679e0e45e46e651dc7313b4bd431d22e0145811249e11df96672e572e71c: Status 404 returned error can't find the container with id d48f679e0e45e46e651dc7313b4bd431d22e0145811249e11df96672e572e71c Jul 08 21:17:05.810217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.810195 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:17:05.812759 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:05.812726 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9da108_4ac6_489c_8d70_94c2139c0635.slice/crio-159a5ad5dc91c699aac52d61bed04980832d512add12b1f4e7dc4a3faf2fb996 WatchSource:0}: Error finding container 159a5ad5dc91c699aac52d61bed04980832d512add12b1f4e7dc4a3faf2fb996: Status 404 returned error can't find the container with id 159a5ad5dc91c699aac52d61bed04980832d512add12b1f4e7dc4a3faf2fb996 Jul 08 21:17:05.841300 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.841273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" event={"ID":"e06251dc-8496-4ac1-8197-78be226c9788","Type":"ContainerStarted","Data":"d48f679e0e45e46e651dc7313b4bd431d22e0145811249e11df96672e572e71c"} Jul 08 21:17:05.843279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.843220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" event={"ID":"2e06d0a8-7e51-4db1-87f6-078aaf947050","Type":"ContainerStarted","Data":"3915e3e937d8d98158067c385c0f040bd241acfa40d668d324930e416c6cb793"} Jul 08 21:17:05.845335 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.845299 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerName="manager" containerID="cri-o://6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383" gracePeriod=2 Jul 08 21:17:05.845870 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.845733 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.846802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.846785 2565 scope.go:117] "RemoveContainer" containerID="ef5dad525ed6c96f44f2dbc95c81d9412613546e92ad5093a20fd188be10c3d5" Jul 08 21:17:05.846802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.846796 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.848548 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.848444 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.849394 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.849372 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" event={"ID":"fc9da108-4ac6-489c-8d70-94c2139c0635","Type":"ContainerStarted","Data":"159a5ad5dc91c699aac52d61bed04980832d512add12b1f4e7dc4a3faf2fb996"} Jul 08 21:17:05.851159 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.851139 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.855765 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.855691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" event={"ID":"f04e6e8b-8e33-4c52-89a5-ceadbe32996b","Type":"ContainerStarted","Data":"bfcbc883bdbef49d7382fa73f8d670e6b7b652b5af3ce55b2f6d9854eab0178c"} Jul 08 21:17:05.855765 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.855733 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" event={"ID":"f04e6e8b-8e33-4c52-89a5-ceadbe32996b","Type":"ContainerStarted","Data":"7aaab41621996232c922fa8de6779c734c2d458fc361c7aecffe8237498d6df2"} Jul 08 21:17:05.856016 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.855975 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:17:05.865420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.865393 2565 scope.go:117] "RemoveContainer" containerID="6dca9cf303619c674546e5546233b633c39f0258f5e1f58ccdd511b10e0126dc" Jul 08 21:17:05.867115 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.867080 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7w5x" podStartSLOduration=1.628196959 podStartE2EDuration="25.867066911s" totalImagesPullingTime="24.238869952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:05.864819127 +0000 UTC m=+661.436563989" watchObservedRunningTime="2026-07-08 21:17:05.867066911 +0000 UTC m=+661.438811769" Jul 08 21:17:05.867185 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.867134 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:05.887092 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:05.887044 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" podStartSLOduration=14.887008797 podStartE2EDuration="14.887008797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:05.885794548 +0000 UTC m=+661.457539408" watchObservedRunningTime="2026-07-08 21:17:05.887008797 +0000 UTC m=+661.458753660" Jul 08 21:17:05.887817 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.887790 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.889606 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.889575 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:05.901835 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.901814 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:05.903796 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.903771 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:05.905523 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:05.905502 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:06.076357 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.076335 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.078494 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.078471 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:06.080373 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.080353 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.082286 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.082264 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:06.156194 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.156168 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume\") pod \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " Jul 08 21:17:06.156348 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.156233 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfk4l\" (UniqueName: \"kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l\") pod \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\" (UID: \"aa2e283d-f7f2-47f3-9934-4b9ae3818bb6\") " Jul 08 21:17:06.156535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.156511 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" (UID: "aa2e283d-f7f2-47f3-9934-4b9ae3818bb6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:17:06.158134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.158115 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l" (OuterVolumeSpecName: "kube-api-access-zfk4l") pod "aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" (UID: "aa2e283d-f7f2-47f3-9934-4b9ae3818bb6"). InnerVolumeSpecName "kube-api-access-zfk4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:17:06.257339 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.257273 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:06.257339 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.257300 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfk4l\" (UniqueName: \"kubernetes.io/projected/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6-kube-api-access-zfk4l\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:06.845910 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.845862 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.45:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 21:17:06.862978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.862944 2565 generic.go:356] "Generic (PLEG): container finished" podID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" containerID="6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383" exitCode=0 Jul 08 21:17:06.863117 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.863016 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.863117 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.863050 2565 scope.go:117] "RemoveContainer" containerID="6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383" Jul 08 21:17:06.865839 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.865812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" event={"ID":"fc9da108-4ac6-489c-8d70-94c2139c0635","Type":"ContainerStarted","Data":"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da"} Jul 08 21:17:06.865992 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.865967 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:17:06.866430 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.866403 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:06.868604 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.868566 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.868893 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.868870 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" event={"ID":"e06251dc-8496-4ac1-8197-78be226c9788","Type":"ContainerStarted","Data":"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c"} Jul 08 21:17:06.869016 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.868941 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:17:06.870656 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.870633 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:06.872756 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.872644 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:06.872841 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.872773 2565 scope.go:117] "RemoveContainer" containerID="6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383" Jul 08 21:17:06.873025 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.873007 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383\": container with ID starting with 6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383 not found: ID does not exist" containerID="6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383" Jul 08 21:17:06.873066 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.873033 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383"} err="failed to get container status \"6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383\": rpc error: code = NotFound desc = could not find container \"6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383\": container with ID starting with 6a347dc790794b1a1934e8276851f944d8a0e5e129d511ed939a64ea74f88383 not found: ID does not exist" Jul 08 21:17:06.893663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.893627 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" podStartSLOduration=15.89361471 podStartE2EDuration="15.89361471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:06.892437868 +0000 UTC m=+662.464182725" watchObservedRunningTime="2026-07-08 21:17:06.89361471 +0000 UTC m=+662.465359568" Jul 08 21:17:06.894389 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.894367 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:06.896075 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.896047 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.914348 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.914308 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" podStartSLOduration=15.914293665 podStartE2EDuration="15.914293665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:06.913264376 +0000 UTC m=+662.485009232" watchObservedRunningTime="2026-07-08 21:17:06.914293665 +0000 UTC m=+662.486038525" Jul 08 21:17:06.915317 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.915297 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-28dgd\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-28dgd" Jul 08 21:17:06.917079 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.917054 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qw445\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qw445" Jul 08 21:17:06.918932 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:06.918910 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dqzhj\" is forbidden: User \"system:node:ip-10-0-128-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-213.ec2.internal' and this object" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dqzhj" Jul 08 21:17:06.950545 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.950525 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a2d446-bbcd-400b-9b1c-64c8428dc004" path="/var/lib/kubelet/pods/24a2d446-bbcd-400b-9b1c-64c8428dc004/volumes" Jul 08 21:17:06.950876 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.950863 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa2e283d-f7f2-47f3-9934-4b9ae3818bb6" path="/var/lib/kubelet/pods/aa2e283d-f7f2-47f3-9934-4b9ae3818bb6/volumes" Jul 08 21:17:06.951141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:06.951130 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d92d954f-4085-40b5-80e4-cd9d94af375d" path="/var/lib/kubelet/pods/d92d954f-4085-40b5-80e4-cd9d94af375d/volumes" Jul 08 21:17:16.871025 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:16.870994 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nxghj" Jul 08 21:17:17.877787 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:17.877753 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:17:17.878210 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:17.877817 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:17:17.947935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:17.947908 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:17:17.948110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:17.948089 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" podUID="e06251dc-8496-4ac1-8197-78be226c9788" containerName="manager" containerID="cri-o://725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c" gracePeriod=10 Jul 08 21:17:18.150654 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.150574 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:17:18.153478 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.153460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.167620 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.167594 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:17:18.209877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.209854 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:17:18.258275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258227 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sttg\" (UniqueName: \"kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg\") pod \"e06251dc-8496-4ac1-8197-78be226c9788\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " Jul 08 21:17:18.258406 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258354 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume\") pod \"e06251dc-8496-4ac1-8197-78be226c9788\" (UID: \"e06251dc-8496-4ac1-8197-78be226c9788\") " Jul 08 21:17:18.258529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258513 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.258645 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphs8\" (UniqueName: \"kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.258704 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258641 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e06251dc-8496-4ac1-8197-78be226c9788" (UID: "e06251dc-8496-4ac1-8197-78be226c9788"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:17:18.258746 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.258737 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06251dc-8496-4ac1-8197-78be226c9788-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:18.260253 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.260219 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg" (OuterVolumeSpecName: "kube-api-access-5sttg") pod "e06251dc-8496-4ac1-8197-78be226c9788" (UID: "e06251dc-8496-4ac1-8197-78be226c9788"). InnerVolumeSpecName "kube-api-access-5sttg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:17:18.359882 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.359856 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphs8\" (UniqueName: \"kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.359985 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.359915 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.359985 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.359946 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sttg\" (UniqueName: \"kubernetes.io/projected/e06251dc-8496-4ac1-8197-78be226c9788-kube-api-access-5sttg\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:18.360221 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.360207 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.370091 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.370064 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphs8\" (UniqueName: \"kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8\") pod \"kuadrant-operator-controller-manager-86878c769f-jb9db\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.464959 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.464910 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.606135 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.606109 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:17:18.607220 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:18.607197 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c19b9f_117b_4866_bc9f_505bbcd78b44.slice/crio-c9914a057aad9e4d66875018772a8998e7b235f6e43621a49c2d2750d3e6918f WatchSource:0}: Error finding container c9914a057aad9e4d66875018772a8998e7b235f6e43621a49c2d2750d3e6918f: Status 404 returned error can't find the container with id c9914a057aad9e4d66875018772a8998e7b235f6e43621a49c2d2750d3e6918f Jul 08 21:17:18.936336 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.936290 2565 generic.go:356] "Generic (PLEG): container finished" podID="e06251dc-8496-4ac1-8197-78be226c9788" containerID="725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c" exitCode=0 Jul 08 21:17:18.936336 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.936334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" Jul 08 21:17:18.936853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.936426 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" event={"ID":"e06251dc-8496-4ac1-8197-78be226c9788","Type":"ContainerDied","Data":"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c"} Jul 08 21:17:18.936853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.936494 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4" event={"ID":"e06251dc-8496-4ac1-8197-78be226c9788","Type":"ContainerDied","Data":"d48f679e0e45e46e651dc7313b4bd431d22e0145811249e11df96672e572e71c"} Jul 08 21:17:18.936853 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.936539 2565 scope.go:117] "RemoveContainer" containerID="725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c" Jul 08 21:17:18.938705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.938678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" event={"ID":"52c19b9f-117b-4866-bc9f-505bbcd78b44","Type":"ContainerStarted","Data":"da5197bf811dd6c9f501bb0bb67a16a8be16d8228ad300be98dff4dcbc396a13"} Jul 08 21:17:18.938831 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.938727 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" event={"ID":"52c19b9f-117b-4866-bc9f-505bbcd78b44","Type":"ContainerStarted","Data":"c9914a057aad9e4d66875018772a8998e7b235f6e43621a49c2d2750d3e6918f"} Jul 08 21:17:18.938831 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.938760 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:18.947408 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.947390 2565 scope.go:117] "RemoveContainer" containerID="725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c" Jul 08 21:17:18.947655 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:18.947637 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c\": container with ID starting with 725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c not found: ID does not exist" containerID="725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c" Jul 08 21:17:18.947701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.947663 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c"} err="failed to get container status \"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c\": rpc error: code = NotFound desc = could not find container \"725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c\": container with ID starting with 725db181475226ef8700f9eb329f4f0a7efcfc1644e0ba634881b0e11ea4104c not found: ID does not exist" Jul 08 21:17:18.955802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.955763 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" podStartSLOduration=0.955749582 podStartE2EDuration="955.749582ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:18.954897704 +0000 UTC m=+674.526642563" watchObservedRunningTime="2026-07-08 21:17:18.955749582 +0000 UTC m=+674.527494440" Jul 08 21:17:18.969406 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.969378 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:17:18.976273 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:18.976251 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xmqm4"] Jul 08 21:17:20.954963 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:20.954927 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e06251dc-8496-4ac1-8197-78be226c9788" path="/var/lib/kubelet/pods/e06251dc-8496-4ac1-8197-78be226c9788/volumes" Jul 08 21:17:22.570352 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:22.570308 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:23.308659 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:23.308619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:29.947283 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:29.947227 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:17:29.989575 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:29.989527 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:17:29.989849 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:29.989744 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" podUID="fc9da108-4ac6-489c-8d70-94c2139c0635" containerName="manager" containerID="cri-o://5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da" gracePeriod=10 Jul 08 21:17:30.974904 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:30.974835 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:17:31.010652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.010619 2565 generic.go:356] "Generic (PLEG): container finished" podID="fc9da108-4ac6-489c-8d70-94c2139c0635" containerID="5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da" exitCode=0 Jul 08 21:17:31.010916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.010657 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" Jul 08 21:17:31.010916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.010692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" event={"ID":"fc9da108-4ac6-489c-8d70-94c2139c0635","Type":"ContainerDied","Data":"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da"} Jul 08 21:17:31.010916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.010742 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf" event={"ID":"fc9da108-4ac6-489c-8d70-94c2139c0635","Type":"ContainerDied","Data":"159a5ad5dc91c699aac52d61bed04980832d512add12b1f4e7dc4a3faf2fb996"} Jul 08 21:17:31.010916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.010764 2565 scope.go:117] "RemoveContainer" containerID="5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da" Jul 08 21:17:31.020171 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.020145 2565 scope.go:117] "RemoveContainer" containerID="5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da" Jul 08 21:17:31.020524 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:17:31.020505 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da\": container with ID starting with 5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da not found: ID does not exist" containerID="5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da" Jul 08 21:17:31.020593 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.020537 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da"} err="failed to get container status \"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da\": rpc error: code = NotFound desc = could not find container \"5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da\": container with ID starting with 5ad18db48231f2904aa40a78ac75fdb0a8f55bcbc28fc75d0a9f90a3a77d13da not found: ID does not exist" Jul 08 21:17:31.064592 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.064570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbslq\" (UniqueName: \"kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq\") pod \"fc9da108-4ac6-489c-8d70-94c2139c0635\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " Jul 08 21:17:31.064682 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.064633 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume\") pod \"fc9da108-4ac6-489c-8d70-94c2139c0635\" (UID: \"fc9da108-4ac6-489c-8d70-94c2139c0635\") " Jul 08 21:17:31.064968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.064948 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fc9da108-4ac6-489c-8d70-94c2139c0635" (UID: "fc9da108-4ac6-489c-8d70-94c2139c0635"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:17:31.066645 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.066627 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq" (OuterVolumeSpecName: "kube-api-access-kbslq") pod "fc9da108-4ac6-489c-8d70-94c2139c0635" (UID: "fc9da108-4ac6-489c-8d70-94c2139c0635"). InnerVolumeSpecName "kube-api-access-kbslq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:17:31.165903 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.165880 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9da108-4ac6-489c-8d70-94c2139c0635-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:31.165903 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.165900 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbslq\" (UniqueName: \"kubernetes.io/projected/fc9da108-4ac6-489c-8d70-94c2139c0635-kube-api-access-kbslq\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:17:31.332449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.332426 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:17:31.335802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:31.335782 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mb6gf"] Jul 08 21:17:32.951301 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:32.951271 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9da108-4ac6-489c-8d70-94c2139c0635" path="/var/lib/kubelet/pods/fc9da108-4ac6-489c-8d70-94c2139c0635/volumes" Jul 08 21:17:33.347503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:33.347472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:34.325222 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325182 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm"] Jul 08 21:17:34.325806 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325785 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9da108-4ac6-489c-8d70-94c2139c0635" containerName="manager" Jul 08 21:17:34.325806 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325807 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9da108-4ac6-489c-8d70-94c2139c0635" containerName="manager" Jul 08 21:17:34.325925 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325820 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e06251dc-8496-4ac1-8197-78be226c9788" containerName="manager" Jul 08 21:17:34.325925 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325827 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e06251dc-8496-4ac1-8197-78be226c9788" containerName="manager" Jul 08 21:17:34.326030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325925 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9da108-4ac6-489c-8d70-94c2139c0635" containerName="manager" Jul 08 21:17:34.326030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.325936 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e06251dc-8496-4ac1-8197-78be226c9788" containerName="manager" Jul 08 21:17:34.348678 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.348641 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:34.486566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.486530 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:34.615820 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.615751 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm"] Jul 08 21:17:34.615945 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.615860 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.618601 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.618577 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-t4qqk\"" Jul 08 21:17:34.695371 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695490 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695490 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695490 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv24v\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-kube-api-access-tv24v\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695540 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695555 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.695634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.695571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796261 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796382 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv24v\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-kube-api-access-tv24v\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796382 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796382 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796382 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796407 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796757 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796650 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796757 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.796929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.796908 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.797036 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.797014 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.797119 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.797086 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.798704 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.798678 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.798796 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.798780 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.804217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.804194 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv24v\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-kube-api-access-tv24v\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.804323 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.804294 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b1fbf7-8b5f-4696-9f78-aed53988711b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jp7fm\" (UID: \"c1b1fbf7-8b5f-4696-9f78-aed53988711b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:34.926368 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:34.926305 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:35.155524 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:35.155497 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm"] Jul 08 21:17:35.157593 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:35.157569 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b1fbf7_8b5f_4696_9f78_aed53988711b.slice/crio-23ba907e942863c0c15f9f1679d97c30d69c4296af029dde360a5cf3f2f0373c WatchSource:0}: Error finding container 23ba907e942863c0c15f9f1679d97c30d69c4296af029dde360a5cf3f2f0373c: Status 404 returned error can't find the container with id 23ba907e942863c0c15f9f1679d97c30d69c4296af029dde360a5cf3f2f0373c Jul 08 21:17:35.160180 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:35.160151 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:17:35.160274 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:35.160219 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:17:35.160274 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:35.160268 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:17:36.043944 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:36.043914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" event={"ID":"c1b1fbf7-8b5f-4696-9f78-aed53988711b","Type":"ContainerStarted","Data":"fe9d8763556d274fddfd628a1e8ed17ceaa71f907b7ce9571cbaab355556fba3"} Jul 08 21:17:36.044400 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:36.043958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" event={"ID":"c1b1fbf7-8b5f-4696-9f78-aed53988711b","Type":"ContainerStarted","Data":"23ba907e942863c0c15f9f1679d97c30d69c4296af029dde360a5cf3f2f0373c"} Jul 08 21:17:36.064728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:36.064688 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" podStartSLOduration=2.064675267 podStartE2EDuration="2.064675267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:36.063404322 +0000 UTC m=+691.635149180" watchObservedRunningTime="2026-07-08 21:17:36.064675267 +0000 UTC m=+691.636420126" Jul 08 21:17:36.927204 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:36.927175 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:36.931730 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:36.931708 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:37.049109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:37.049078 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:37.049830 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:37.049813 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:17:42.855354 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:42.855323 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:44.454661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:44.454622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:46.827968 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:46.827933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:51.578025 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.577989 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:17:51.593145 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.593119 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:17:51.593310 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.593209 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.594907 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.594875 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:51.595522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.595506 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 21:17:51.681518 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.681489 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:17:51.683837 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.683805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:51.736344 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.736308 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:51.737456 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.737437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.737527 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.737465 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6gms\" (UniqueName: \"kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.838608 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.838531 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.838608 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.838561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6gms\" (UniqueName: \"kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.839143 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.839125 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.846505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.846478 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6gms\" (UniqueName: \"kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms\") pod \"limitador-limitador-64c94fd64f-nt2b7\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:51.904969 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:51.904942 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:52.049045 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:52.049017 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:17:52.050981 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:17:52.050955 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc745cd09_49e4_43d4_8581_a66c6c0e1554.slice/crio-6a8a4df21027a12eadbf79731b2259c896985caf3bfba18878ee4c73d266da33 WatchSource:0}: Error finding container 6a8a4df21027a12eadbf79731b2259c896985caf3bfba18878ee4c73d266da33: Status 404 returned error can't find the container with id 6a8a4df21027a12eadbf79731b2259c896985caf3bfba18878ee4c73d266da33 Jul 08 21:17:52.151263 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:52.151175 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" event={"ID":"c745cd09-49e4-43d4-8581-a66c6c0e1554","Type":"ContainerStarted","Data":"6a8a4df21027a12eadbf79731b2259c896985caf3bfba18878ee4c73d266da33"} Jul 08 21:17:52.412210 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:52.412130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:52.469074 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:52.469043 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:17:55.172393 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:55.172320 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" event={"ID":"c745cd09-49e4-43d4-8581-a66c6c0e1554","Type":"ContainerStarted","Data":"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88"} Jul 08 21:17:55.172730 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:55.172455 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:17:55.189161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:17:55.189121 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" podStartSLOduration=1.403544564 podStartE2EDuration="4.189106633s" totalImagesPullingTime="2.785562069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:17:55.187089067 +0000 UTC m=+710.758833922" watchObservedRunningTime="2026-07-08 21:17:55.189106633 +0000 UTC m=+710.760851492" Jul 08 21:18:06.179762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:06.179726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:18:08.933123 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:08.933077 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:18:08.933561 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:08.933313 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" podUID="c745cd09-49e4-43d4-8581-a66c6c0e1554" containerName="limitador" containerID="cri-o://b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88" gracePeriod=30 Jul 08 21:18:09.487549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.487521 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:18:09.591199 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.591175 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file\") pod \"c745cd09-49e4-43d4-8581-a66c6c0e1554\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " Jul 08 21:18:09.591359 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.591216 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6gms\" (UniqueName: \"kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms\") pod \"c745cd09-49e4-43d4-8581-a66c6c0e1554\" (UID: \"c745cd09-49e4-43d4-8581-a66c6c0e1554\") " Jul 08 21:18:09.591518 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.591495 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file" (OuterVolumeSpecName: "config-file") pod "c745cd09-49e4-43d4-8581-a66c6c0e1554" (UID: "c745cd09-49e4-43d4-8581-a66c6c0e1554"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:18:09.593179 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.593160 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms" (OuterVolumeSpecName: "kube-api-access-b6gms") pod "c745cd09-49e4-43d4-8581-a66c6c0e1554" (UID: "c745cd09-49e4-43d4-8581-a66c6c0e1554"). InnerVolumeSpecName "kube-api-access-b6gms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:18:09.692785 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.692760 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c745cd09-49e4-43d4-8581-a66c6c0e1554-config-file\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:18:09.692785 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:09.692783 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6gms\" (UniqueName: \"kubernetes.io/projected/c745cd09-49e4-43d4-8581-a66c6c0e1554-kube-api-access-b6gms\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:18:10.254144 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.254112 2565 generic.go:356] "Generic (PLEG): container finished" podID="c745cd09-49e4-43d4-8581-a66c6c0e1554" containerID="b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88" exitCode=0 Jul 08 21:18:10.254554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.254150 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" Jul 08 21:18:10.254554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.254190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" event={"ID":"c745cd09-49e4-43d4-8581-a66c6c0e1554","Type":"ContainerDied","Data":"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88"} Jul 08 21:18:10.254554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.254252 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nt2b7" event={"ID":"c745cd09-49e4-43d4-8581-a66c6c0e1554","Type":"ContainerDied","Data":"6a8a4df21027a12eadbf79731b2259c896985caf3bfba18878ee4c73d266da33"} Jul 08 21:18:10.254554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.254271 2565 scope.go:117] "RemoveContainer" containerID="b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88" Jul 08 21:18:10.264224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.264207 2565 scope.go:117] "RemoveContainer" containerID="b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88" Jul 08 21:18:10.264491 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:18:10.264474 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88\": container with ID starting with b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88 not found: ID does not exist" containerID="b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88" Jul 08 21:18:10.264550 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.264500 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88"} err="failed to get container status \"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88\": rpc error: code = NotFound desc = could not find container \"b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88\": container with ID starting with b2033bb5c843ae76e27463f56c0a1be231cc0e6b8f7174a1e12df2dd46184f88 not found: ID does not exist" Jul 08 21:18:10.275902 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.275862 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:18:10.277771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.277753 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nt2b7"] Jul 08 21:18:10.949949 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:10.949908 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c745cd09-49e4-43d4-8581-a66c6c0e1554" path="/var/lib/kubelet/pods/c745cd09-49e4-43d4-8581-a66c6c0e1554/volumes" Jul 08 21:18:13.836767 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:13.836730 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:13.935754 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:13.935719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:14.326522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.326494 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx"] Jul 08 21:18:14.327164 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.327149 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c745cd09-49e4-43d4-8581-a66c6c0e1554" containerName="limitador" Jul 08 21:18:14.327216 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.327169 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c745cd09-49e4-43d4-8581-a66c6c0e1554" containerName="limitador" Jul 08 21:18:14.327637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.327287 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c745cd09-49e4-43d4-8581-a66c6c0e1554" containerName="limitador" Jul 08 21:18:14.330533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.330517 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.333094 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.333074 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 21:18:14.333225 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.333076 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bthw5\"" Jul 08 21:18:14.333225 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.333192 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 21:18:14.334133 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.334114 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 21:18:14.339717 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.339693 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx"] Jul 08 21:18:14.355537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.355506 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:14.433771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.433747 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0c1469e-bbc5-4a59-8044-01257d6e09d5-data\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.433891 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.433781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrhh2\" (UniqueName: \"kubernetes.io/projected/c0c1469e-bbc5-4a59-8044-01257d6e09d5-kube-api-access-mrhh2\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.535313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.535289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0c1469e-bbc5-4a59-8044-01257d6e09d5-data\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.535427 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.535334 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrhh2\" (UniqueName: \"kubernetes.io/projected/c0c1469e-bbc5-4a59-8044-01257d6e09d5-kube-api-access-mrhh2\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.535621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.535604 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0c1469e-bbc5-4a59-8044-01257d6e09d5-data\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.542663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.542643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrhh2\" (UniqueName: \"kubernetes.io/projected/c0c1469e-bbc5-4a59-8044-01257d6e09d5-kube-api-access-mrhh2\") pod \"postgres-69fb4594cf-rdvmx\" (UID: \"c0c1469e-bbc5-4a59-8044-01257d6e09d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.642338 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.642282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:14.772127 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:14.772101 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx"] Jul 08 21:18:14.774039 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:18:14.774017 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c1469e_bbc5_4a59_8044_01257d6e09d5.slice/crio-2c841455d0aa659d81e8bf6331bbd54935609b85f9d6296cfce697a7ced1726e WatchSource:0}: Error finding container 2c841455d0aa659d81e8bf6331bbd54935609b85f9d6296cfce697a7ced1726e: Status 404 returned error can't find the container with id 2c841455d0aa659d81e8bf6331bbd54935609b85f9d6296cfce697a7ced1726e Jul 08 21:18:15.286194 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:15.286110 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" event={"ID":"c0c1469e-bbc5-4a59-8044-01257d6e09d5","Type":"ContainerStarted","Data":"2c841455d0aa659d81e8bf6331bbd54935609b85f9d6296cfce697a7ced1726e"} Jul 08 21:18:19.315131 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:19.315102 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" event={"ID":"c0c1469e-bbc5-4a59-8044-01257d6e09d5","Type":"ContainerStarted","Data":"97ec5870ce12a852263d647072f7454fa2ab1fe084f029d82e0a0b08f45fd1c3"} Jul 08 21:18:19.315475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:19.315168 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:19.329181 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:19.329137 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" podStartSLOduration=0.855597309 podStartE2EDuration="5.32912248s" totalImagesPullingTime="4.473525171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:18:19.32796161 +0000 UTC m=+734.899706473" watchObservedRunningTime="2026-07-08 21:18:19.32912248 +0000 UTC m=+734.900867339" Jul 08 21:18:25.352887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:25.352859 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rdvmx" Jul 08 21:18:25.661901 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:25.661814 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:25.888319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:25.888291 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:32.605046 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:32.605013 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:32.699109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:32.699069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:32.752087 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:32.752048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:35.486571 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:35.486531 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:18:35.589147 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:18:35.589111 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:06.941589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:06.941557 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:07.363320 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:07.363291 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:07.571595 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:07.571555 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:07.782265 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:07.782209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:07.888265 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:07.888209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:15.628576 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.628542 2565 generic.go:356] "Generic (PLEG): container finished" podID="c1b1fbf7-8b5f-4696-9f78-aed53988711b" containerID="fe9d8763556d274fddfd628a1e8ed17ceaa71f907b7ce9571cbaab355556fba3" exitCode=0 Jul 08 21:19:15.629027 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.628611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" event={"ID":"c1b1fbf7-8b5f-4696-9f78-aed53988711b","Type":"ContainerDied","Data":"fe9d8763556d274fddfd628a1e8ed17ceaa71f907b7ce9571cbaab355556fba3"} Jul 08 21:19:15.629027 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.628981 2565 scope.go:117] "RemoveContainer" containerID="fe9d8763556d274fddfd628a1e8ed17ceaa71f907b7ce9571cbaab355556fba3" Jul 08 21:19:15.629936 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.629911 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:19:15.630003 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.629990 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:19:15.630044 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:15.630025 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:19:16.639674 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:16.639629 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" event={"ID":"c1b1fbf7-8b5f-4696-9f78-aed53988711b","Type":"ContainerStarted","Data":"c7774208373afaf5254941a471bd614d704a5a415fcec9f6d9105ff2d23da70b"} Jul 08 21:19:16.927010 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:16.926930 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:19:16.927155 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:16.927114 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jp7fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 21:19:16.927197 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:16.927162 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" podUID="c1b1fbf7-8b5f-4696-9f78-aed53988711b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 21:19:17.351819 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:17.351784 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:17.455800 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:17.455769 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:17.927578 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:17.927543 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jp7fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 21:19:17.927951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:17.927600 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" podUID="c1b1fbf7-8b5f-4696-9f78-aed53988711b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 21:19:18.926664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:18.926630 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jp7fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 21:19:18.926821 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:18.926691 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" podUID="c1b1fbf7-8b5f-4696-9f78-aed53988711b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 21:19:19.927200 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:19.927163 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jp7fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 21:19:19.927593 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:19.927220 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" podUID="c1b1fbf7-8b5f-4696-9f78-aed53988711b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 21:19:20.930325 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:20.930290 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:19:20.930692 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:20.930568 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:19:20.931322 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:20.931301 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jp7fm" Jul 08 21:19:29.810032 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.809960 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw"] Jul 08 21:19:29.812851 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.812834 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.815263 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.815213 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 21:19:29.816411 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.816389 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 21:19:29.816533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.816513 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2tfz7\"" Jul 08 21:19:29.816606 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.816518 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 21:19:29.822981 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.822957 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw"] Jul 08 21:19:29.835666 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.835626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:29.870510 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.870596 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870521 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.870643 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870593 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsfpm\" (UniqueName: \"kubernetes.io/projected/f284c50a-46a0-45c4-a709-0477cd842442-kube-api-access-rsfpm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.870680 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870638 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.870680 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.870680 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.870674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f284c50a-46a0-45c4-a709-0477cd842442-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.971740 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.971852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.971852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f284c50a-46a0-45c4-a709-0477cd842442-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.971852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971801 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.971852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.972005 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.971879 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsfpm\" (UniqueName: \"kubernetes.io/projected/f284c50a-46a0-45c4-a709-0477cd842442-kube-api-access-rsfpm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.972222 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.972186 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.972222 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.972199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.972398 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.972257 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.974020 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.973996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f284c50a-46a0-45c4-a709-0477cd842442-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.974455 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.974430 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f284c50a-46a0-45c4-a709-0477cd842442-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:29.978949 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:29.978929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsfpm\" (UniqueName: \"kubernetes.io/projected/f284c50a-46a0-45c4-a709-0477cd842442-kube-api-access-rsfpm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw\" (UID: \"f284c50a-46a0-45c4-a709-0477cd842442\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:30.125469 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:30.125395 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:30.262119 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:30.262080 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw"] Jul 08 21:19:30.263389 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:19:30.263365 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf284c50a_46a0_45c4_a709_0477cd842442.slice/crio-72bacbd6684b023997ff072396a3a44d924c35dbf261727f8c0dcf41defa401e WatchSource:0}: Error finding container 72bacbd6684b023997ff072396a3a44d924c35dbf261727f8c0dcf41defa401e: Status 404 returned error can't find the container with id 72bacbd6684b023997ff072396a3a44d924c35dbf261727f8c0dcf41defa401e Jul 08 21:19:30.265223 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:30.265205 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:19:30.726940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:30.726893 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" event={"ID":"f284c50a-46a0-45c4-a709-0477cd842442","Type":"ContainerStarted","Data":"72bacbd6684b023997ff072396a3a44d924c35dbf261727f8c0dcf41defa401e"} Jul 08 21:19:31.734095 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:31.734061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" event={"ID":"f284c50a-46a0-45c4-a709-0477cd842442","Type":"ContainerStarted","Data":"108af67e2de8b18f7a045fdf76b22033dd340aeb8775202873680f37ac1f5320"} Jul 08 21:19:31.751101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:31.751052 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" podStartSLOduration=1.759205849 podStartE2EDuration="2.751037741s" totalImagesPullingTime="991.831892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:31.749832332 +0000 UTC m=+807.321577191" watchObservedRunningTime="2026-07-08 21:19:31.751037741 +0000 UTC m=+807.322782641" Jul 08 21:19:33.505646 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:33.505613 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:36.215143 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.215112 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx"] Jul 08 21:19:36.217790 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.217774 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.221164 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.221138 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 21:19:36.227698 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.227671 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx"] Jul 08 21:19:36.330748 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.330899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.330899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330769 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.330899 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.331010 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjnkp\" (UniqueName: \"kubernetes.io/projected/efd88a8f-916a-4a50-8516-da51c424487d-kube-api-access-hjnkp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.331010 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.330982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efd88a8f-916a-4a50-8516-da51c424487d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.422553 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.422518 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:36.431621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjnkp\" (UniqueName: \"kubernetes.io/projected/efd88a8f-916a-4a50-8516-da51c424487d-kube-api-access-hjnkp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.431721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431652 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efd88a8f-916a-4a50-8516-da51c424487d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.431721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.431721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.431873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.431873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.431746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.432131 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.432110 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.432199 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.432125 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.432199 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.432181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.433966 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.433943 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efd88a8f-916a-4a50-8516-da51c424487d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.434152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.434130 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efd88a8f-916a-4a50-8516-da51c424487d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.441659 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.441635 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjnkp\" (UniqueName: \"kubernetes.io/projected/efd88a8f-916a-4a50-8516-da51c424487d-kube-api-access-hjnkp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhczx\" (UID: \"efd88a8f-916a-4a50-8516-da51c424487d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.529386 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.529365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:36.661395 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.661366 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx"] Jul 08 21:19:36.663197 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:19:36.663175 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd88a8f_916a_4a50_8516_da51c424487d.slice/crio-e593a78e788c503b1b07b8990836c799c1f8bfc5c00b92aade47d65f17e71137 WatchSource:0}: Error finding container e593a78e788c503b1b07b8990836c799c1f8bfc5c00b92aade47d65f17e71137: Status 404 returned error can't find the container with id e593a78e788c503b1b07b8990836c799c1f8bfc5c00b92aade47d65f17e71137 Jul 08 21:19:36.762595 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:36.762562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" event={"ID":"efd88a8f-916a-4a50-8516-da51c424487d","Type":"ContainerStarted","Data":"e593a78e788c503b1b07b8990836c799c1f8bfc5c00b92aade47d65f17e71137"} Jul 08 21:19:37.769227 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:37.769194 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" event={"ID":"efd88a8f-916a-4a50-8516-da51c424487d","Type":"ContainerStarted","Data":"6cd2999f918e1854e2d09b2d02f6820ef409184e5861e1f1d00822bba69c1d6a"} Jul 08 21:19:37.786290 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:37.786231 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" podStartSLOduration=1.618684688 podStartE2EDuration="1.786215539s" totalImagesPullingTime="167.530851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:37.784994951 +0000 UTC m=+813.356739810" watchObservedRunningTime="2026-07-08 21:19:37.786215539 +0000 UTC m=+813.357960447" Jul 08 21:19:40.126342 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:40.126300 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:40.126749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:40.126362 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:40.138823 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:40.138797 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:40.795652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:40.795624 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw" Jul 08 21:19:44.017291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.017256 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw"] Jul 08 21:19:44.020735 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.020707 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.023148 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.023124 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 21:19:44.028760 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.028735 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw"] Jul 08 21:19:44.043026 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.042999 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:19:44.101309 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.101425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101319 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.101425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.101425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a668954-773d-49bc-948d-0e219dc42eb6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.101547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101447 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvzql\" (UniqueName: \"kubernetes.io/projected/7a668954-773d-49bc-948d-0e219dc42eb6-kube-api-access-tvzql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.101547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.101483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202681 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202654 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a668954-773d-49bc-948d-0e219dc42eb6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvzql\" (UniqueName: \"kubernetes.io/projected/7a668954-773d-49bc-948d-0e219dc42eb6-kube-api-access-tvzql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.202987 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.202814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.203104 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.203086 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.203158 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.203122 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.203196 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.203187 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.205031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.205013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a668954-773d-49bc-948d-0e219dc42eb6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.205485 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.205467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a668954-773d-49bc-948d-0e219dc42eb6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.211134 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.211114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvzql\" (UniqueName: \"kubernetes.io/projected/7a668954-773d-49bc-948d-0e219dc42eb6-kube-api-access-tvzql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t4xtw\" (UID: \"7a668954-773d-49bc-948d-0e219dc42eb6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.333828 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.333803 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:44.466458 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.466431 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw"] Jul 08 21:19:44.467735 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:19:44.467710 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a668954_773d_49bc_948d_0e219dc42eb6.slice/crio-d0188d937c5edc63a6e1f09f084337ff6ea5fe5611b0458cbe5fb979eff64911 WatchSource:0}: Error finding container d0188d937c5edc63a6e1f09f084337ff6ea5fe5611b0458cbe5fb979eff64911: Status 404 returned error can't find the container with id d0188d937c5edc63a6e1f09f084337ff6ea5fe5611b0458cbe5fb979eff64911 Jul 08 21:19:44.807136 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.807103 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" event={"ID":"7a668954-773d-49bc-948d-0e219dc42eb6","Type":"ContainerStarted","Data":"e14171d73b1b06d967fafd282fe92fa85384d50886de676171c39b8f629007ff"} Jul 08 21:19:44.807316 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.807152 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" event={"ID":"7a668954-773d-49bc-948d-0e219dc42eb6","Type":"ContainerStarted","Data":"d0188d937c5edc63a6e1f09f084337ff6ea5fe5611b0458cbe5fb979eff64911"} Jul 08 21:19:44.822996 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:44.822951 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" podStartSLOduration=1.639977795 podStartE2EDuration="1.822937753s" totalImagesPullingTime="182.959958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:44.822149155 +0000 UTC m=+820.393894015" watchObservedRunningTime="2026-07-08 21:19:44.822937753 +0000 UTC m=+820.394682611" Jul 08 21:19:46.529827 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:46.529793 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:46.530329 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:46.529899 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:46.541978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:46.541955 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:46.829192 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:46.829116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhczx" Jul 08 21:19:54.334317 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:54.334280 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:54.334770 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:54.334345 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:54.346659 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:54.346636 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:19:54.876288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:19:54.876261 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t4xtw" Jul 08 21:20:01.628660 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:20:01.628625 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:20:07.447721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:20:07.447685 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:20:11.818951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:20:11.818915 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:20:45.943415 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:20:45.943384 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:20:46.043182 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:20:46.043158 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:21:04.907978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:04.907895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:21:04.911723 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:04.911701 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:21:13.306475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:13.306438 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:21:13.406766 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:13.406719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:21:14.576705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:14.576653 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:21:14.679007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:14.678975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:21:50.485317 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:21:50.485279 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:22:26.574808 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:22:26.574728 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:22:27.946062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:22:27.946029 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:08.786081 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:08.786047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:08.885660 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:08.885623 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:09.021460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:09.021428 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:14.697376 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:14.697332 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:14.766029 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:14.765994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:14.837573 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:14.837529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:14.941484 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:14.941456 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:15.101955 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:15.101911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:21.151050 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:21.151013 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:23:21.230010 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:23:21.229973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:24:14.312855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:24:14.312819 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:24:57.116779 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:24:57.116739 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:25:56.745474 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:25:56.745382 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:25:56.845484 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:25:56.845452 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:26:04.944956 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:26:04.944931 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:26:04.954300 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:26:04.954233 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:26:17.000484 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:26:17.000447 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:27:11.754086 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:11.754048 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:27:11.754672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:11.754318 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" podUID="52c19b9f-117b-4866-bc9f-505bbcd78b44" containerName="manager" containerID="cri-o://da5197bf811dd6c9f501bb0bb67a16a8be16d8228ad300be98dff4dcbc396a13" gracePeriod=10 Jul 08 21:27:14.244902 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.244842 2565 generic.go:356] "Generic (PLEG): container finished" podID="52c19b9f-117b-4866-bc9f-505bbcd78b44" containerID="da5197bf811dd6c9f501bb0bb67a16a8be16d8228ad300be98dff4dcbc396a13" exitCode=0 Jul 08 21:27:14.245188 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.244914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" event={"ID":"52c19b9f-117b-4866-bc9f-505bbcd78b44","Type":"ContainerDied","Data":"da5197bf811dd6c9f501bb0bb67a16a8be16d8228ad300be98dff4dcbc396a13"} Jul 08 21:27:14.614911 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.614888 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:27:14.727375 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.727347 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume\") pod \"52c19b9f-117b-4866-bc9f-505bbcd78b44\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " Jul 08 21:27:14.727375 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.727379 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qphs8\" (UniqueName: \"kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8\") pod \"52c19b9f-117b-4866-bc9f-505bbcd78b44\" (UID: \"52c19b9f-117b-4866-bc9f-505bbcd78b44\") " Jul 08 21:27:14.727728 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.727705 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "52c19b9f-117b-4866-bc9f-505bbcd78b44" (UID: "52c19b9f-117b-4866-bc9f-505bbcd78b44"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:27:14.729252 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.729223 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8" (OuterVolumeSpecName: "kube-api-access-qphs8") pod "52c19b9f-117b-4866-bc9f-505bbcd78b44" (UID: "52c19b9f-117b-4866-bc9f-505bbcd78b44"). InnerVolumeSpecName "kube-api-access-qphs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:27:14.828983 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.828955 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c19b9f-117b-4866-bc9f-505bbcd78b44-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:27:14.828983 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:14.828979 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qphs8\" (UniqueName: \"kubernetes.io/projected/52c19b9f-117b-4866-bc9f-505bbcd78b44-kube-api-access-qphs8\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:27:15.250749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:15.250677 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" event={"ID":"52c19b9f-117b-4866-bc9f-505bbcd78b44","Type":"ContainerDied","Data":"c9914a057aad9e4d66875018772a8998e7b235f6e43621a49c2d2750d3e6918f"} Jul 08 21:27:15.250749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:15.250711 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db" Jul 08 21:27:15.250749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:15.250746 2565 scope.go:117] "RemoveContainer" containerID="da5197bf811dd6c9f501bb0bb67a16a8be16d8228ad300be98dff4dcbc396a13" Jul 08 21:27:15.267951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:15.267928 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:27:15.273507 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:15.273488 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jb9db"] Jul 08 21:27:16.950233 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:27:16.950203 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c19b9f-117b-4866-bc9f-505bbcd78b44" path="/var/lib/kubelet/pods/52c19b9f-117b-4866-bc9f-505bbcd78b44/volumes" Jul 08 21:28:12.984084 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.984050 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc"] Jul 08 21:28:12.984530 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.984515 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c19b9f-117b-4866-bc9f-505bbcd78b44" containerName="manager" Jul 08 21:28:12.984530 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.984531 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c19b9f-117b-4866-bc9f-505bbcd78b44" containerName="manager" Jul 08 21:28:12.984604 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.984600 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c19b9f-117b-4866-bc9f-505bbcd78b44" containerName="manager" Jul 08 21:28:12.988791 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.988775 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:12.991197 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:12.991177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wkt97\"" Jul 08 21:28:13.000678 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.000659 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc"] Jul 08 21:28:13.115193 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.115170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbtw\" (UniqueName: \"kubernetes.io/projected/4ed42850-f404-44d6-a4ae-5fb703b92fff-kube-api-access-7qbtw\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.115307 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.115294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ed42850-f404-44d6-a4ae-5fb703b92fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.216284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.216232 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ed42850-f404-44d6-a4ae-5fb703b92fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.216372 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.216308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbtw\" (UniqueName: \"kubernetes.io/projected/4ed42850-f404-44d6-a4ae-5fb703b92fff-kube-api-access-7qbtw\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.216588 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.216573 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ed42850-f404-44d6-a4ae-5fb703b92fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.224667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.224643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbtw\" (UniqueName: \"kubernetes.io/projected/4ed42850-f404-44d6-a4ae-5fb703b92fff-kube-api-access-7qbtw\") pod \"kuadrant-operator-controller-manager-86878c769f-gjnwc\" (UID: \"4ed42850-f404-44d6-a4ae-5fb703b92fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.299597 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.299578 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.442170 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.442146 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc"] Jul 08 21:28:13.446709 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:28:13.446676 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed42850_f404_44d6_a4ae_5fb703b92fff.slice/crio-e8dc8013451ceed54a1eabc8c4d145fc4bb6739d1ea3272644db9c1178046339 WatchSource:0}: Error finding container e8dc8013451ceed54a1eabc8c4d145fc4bb6739d1ea3272644db9c1178046339: Status 404 returned error can't find the container with id e8dc8013451ceed54a1eabc8c4d145fc4bb6739d1ea3272644db9c1178046339 Jul 08 21:28:13.552088 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.552032 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" event={"ID":"4ed42850-f404-44d6-a4ae-5fb703b92fff","Type":"ContainerStarted","Data":"23805a7b6e1a940b219e482b246754088297ce7571f3b5983e1ea44b04ddf98a"} Jul 08 21:28:13.552088 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.552080 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" event={"ID":"4ed42850-f404-44d6-a4ae-5fb703b92fff","Type":"ContainerStarted","Data":"e8dc8013451ceed54a1eabc8c4d145fc4bb6739d1ea3272644db9c1178046339"} Jul 08 21:28:13.552224 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.552147 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:28:13.571759 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:13.571720 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" podStartSLOduration=1.571705884 podStartE2EDuration="1.571705884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:13.570363673 +0000 UTC m=+1329.142108534" watchObservedRunningTime="2026-07-08 21:28:13.571705884 +0000 UTC m=+1329.143451129" Jul 08 21:28:18.232807 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:18.232773 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:28:24.560290 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:28:24.560260 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gjnwc" Jul 08 21:30:00.148056 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.148023 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:30:00.151480 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.151465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:30:00.153848 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.153819 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4j6cj\"" Jul 08 21:30:00.167284 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.167259 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:30:00.194759 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.194733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hdxc\" (UniqueName: \"kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc\") pod \"maas-api-key-cleanup-29725770-gr2cf\" (UID: \"c1debf31-ee6f-424b-9834-a4df809bd8ea\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:30:00.295607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.295578 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hdxc\" (UniqueName: \"kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc\") pod \"maas-api-key-cleanup-29725770-gr2cf\" (UID: \"c1debf31-ee6f-424b-9834-a4df809bd8ea\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:30:00.303950 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.303926 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hdxc\" (UniqueName: \"kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc\") pod \"maas-api-key-cleanup-29725770-gr2cf\" (UID: \"c1debf31-ee6f-424b-9834-a4df809bd8ea\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:30:00.461997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.461934 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:30:00.592896 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.592861 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:30:00.593113 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:30:00.593083 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1debf31_ee6f_424b_9834_a4df809bd8ea.slice/crio-edff25c1dd9c280a748a038039b88ebeb944a646654d580fced7611671b527c9 WatchSource:0}: Error finding container edff25c1dd9c280a748a038039b88ebeb944a646654d580fced7611671b527c9: Status 404 returned error can't find the container with id edff25c1dd9c280a748a038039b88ebeb944a646654d580fced7611671b527c9 Jul 08 21:30:00.595101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:00.595084 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:30:01.138648 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:01.138610 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" event={"ID":"c1debf31-ee6f-424b-9834-a4df809bd8ea","Type":"ContainerStarted","Data":"edff25c1dd9c280a748a038039b88ebeb944a646654d580fced7611671b527c9"} Jul 08 21:30:08.177109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:08.177074 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" event={"ID":"c1debf31-ee6f-424b-9834-a4df809bd8ea","Type":"ContainerStarted","Data":"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e"} Jul 08 21:30:08.191083 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:08.191041 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" podStartSLOduration=1.7782634800000001 podStartE2EDuration="8.191027382s" totalImagesPullingTime="6.412763902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:30:08.190675471 +0000 UTC m=+1443.762420341" watchObservedRunningTime="2026-07-08 21:30:08.191027382 +0000 UTC m=+1443.762772241" Jul 08 21:30:21.858255 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:21.858212 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:30:21.913834 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:30:21.913806 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:31:00.014898 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:00.014861 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:31:00.015807 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:00.015755 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" podUID="c1debf31-ee6f-424b-9834-a4df809bd8ea" containerName="cleanup" containerID="cri-o://394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e" gracePeriod=30 Jul 08 21:31:04.981846 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:04.981820 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:31:04.991579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:04.991551 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:31:30.204579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.204557 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:31:30.260774 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.260748 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hdxc\" (UniqueName: \"kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc\") pod \"c1debf31-ee6f-424b-9834-a4df809bd8ea\" (UID: \"c1debf31-ee6f-424b-9834-a4df809bd8ea\") " Jul 08 21:31:30.262815 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.262791 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc" (OuterVolumeSpecName: "kube-api-access-5hdxc") pod "c1debf31-ee6f-424b-9834-a4df809bd8ea" (UID: "c1debf31-ee6f-424b-9834-a4df809bd8ea"). InnerVolumeSpecName "kube-api-access-5hdxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:31:30.361661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.361639 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hdxc\" (UniqueName: \"kubernetes.io/projected/c1debf31-ee6f-424b-9834-a4df809bd8ea-kube-api-access-5hdxc\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:31:30.635965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.635887 2565 generic.go:356] "Generic (PLEG): container finished" podID="c1debf31-ee6f-424b-9834-a4df809bd8ea" containerID="394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e" exitCode=137 Jul 08 21:31:30.635965 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.635932 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" Jul 08 21:31:30.636162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.635924 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" event={"ID":"c1debf31-ee6f-424b-9834-a4df809bd8ea","Type":"ContainerDied","Data":"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e"} Jul 08 21:31:30.636162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.636068 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf" event={"ID":"c1debf31-ee6f-424b-9834-a4df809bd8ea","Type":"ContainerDied","Data":"edff25c1dd9c280a748a038039b88ebeb944a646654d580fced7611671b527c9"} Jul 08 21:31:30.636162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.636101 2565 scope.go:117] "RemoveContainer" containerID="394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e" Jul 08 21:31:30.646053 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.646038 2565 scope.go:117] "RemoveContainer" containerID="394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e" Jul 08 21:31:30.646355 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:31:30.646322 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e\": container with ID starting with 394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e not found: ID does not exist" containerID="394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e" Jul 08 21:31:30.646435 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.646360 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e"} err="failed to get container status \"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e\": rpc error: code = NotFound desc = could not find container \"394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e\": container with ID starting with 394d1ef89b99d06d6191d48808638c03acc68e52f3de74fe2275db66adec873e not found: ID does not exist" Jul 08 21:31:30.658146 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.658127 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:31:30.662756 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.662736 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-gr2cf"] Jul 08 21:31:30.950023 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:31:30.949966 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1debf31-ee6f-424b-9834-a4df809bd8ea" path="/var/lib/kubelet/pods/c1debf31-ee6f-424b-9834-a4df809bd8ea/volumes" Jul 08 21:32:37.348667 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:32:37.348628 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:32:37.401371 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:32:37.401342 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.407721 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:33:29.407681 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:50.193898 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:33:50.193860 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:55.621796 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:33:55.621753 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:03.907329 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:03.907283 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:09.076768 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:09.076737 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:12.122941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:12.122900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:12.223263 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:12.223207 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:12.641686 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:12.641651 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:18.451129 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:18.451096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:18.551032 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:18.551003 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:18.667475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:18.667433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:18.731554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:18.731475 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:19.770200 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:19.770166 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:19.846591 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:19.846548 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:19.910401 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:19.910372 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:20.169779 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:20.169746 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:20.433635 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:20.433544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:20.534250 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:20.534215 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.324954 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:22.324916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:24.952168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:24.952137 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:25.160257 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:25.160211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:25.269759 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:25.269672 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:25.371265 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:25.371208 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:26.882233 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:26.882199 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:30.890550 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:30.890510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:30.953052 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:30.953002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:31.073503 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.073470 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:31.074154 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.074134 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1debf31-ee6f-424b-9834-a4df809bd8ea" containerName="cleanup" Jul 08 21:34:31.074203 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.074159 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1debf31-ee6f-424b-9834-a4df809bd8ea" containerName="cleanup" Jul 08 21:34:31.074363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.074348 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1debf31-ee6f-424b-9834-a4df809bd8ea" containerName="cleanup" Jul 08 21:34:31.078181 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.078163 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.080680 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.080655 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-28a93421-openshift-default-dockercfg-4pnv7\"" Jul 08 21:34:31.086144 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.086120 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:31.110434 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.110404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:31.168454 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.168399 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:31.211808 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.211917 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.211917 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211891 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.211996 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.211996 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.211996 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211980 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.212110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.211996 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.212110 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.212032 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.212196 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.212118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z5mw\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.312935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.312912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z5mw\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.312941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.312965 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313017 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313209 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313209 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313209 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313209 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313535 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313644 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313548 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313717 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313638 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313717 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.313876 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.313855 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.315408 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.315385 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.315553 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.315539 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.327054 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.327033 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.327311 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.327293 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z5mw\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw\") pod \"e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.391346 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.391326 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:31.530751 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.530721 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:31.532464 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:34:31.532437 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024489bc_aac7_4cca_ab46_bc7ff9f9ecd6.slice/crio-6dd89e07069a885a3fd6a541582b6390c82d3ca799e8dbb1b5ef1f49fe628b10 WatchSource:0}: Error finding container 6dd89e07069a885a3fd6a541582b6390c82d3ca799e8dbb1b5ef1f49fe628b10: Status 404 returned error can't find the container with id 6dd89e07069a885a3fd6a541582b6390c82d3ca799e8dbb1b5ef1f49fe628b10 Jul 08 21:34:31.535080 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.535051 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:31.535152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.535115 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:31.535152 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.535144 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:31.634755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.634718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerStarted","Data":"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac"} Jul 08 21:34:31.634912 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.634785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerStarted","Data":"6dd89e07069a885a3fd6a541582b6390c82d3ca799e8dbb1b5ef1f49fe628b10"} Jul 08 21:34:31.653400 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:31.653343 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" podStartSLOduration=0.65332357 podStartE2EDuration="653.32357ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:31.651393712 +0000 UTC m=+1707.223138573" watchObservedRunningTime="2026-07-08 21:34:31.65332357 +0000 UTC m=+1707.225068430" Jul 08 21:34:32.226585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.226546 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:32.313153 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.313115 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:32.378771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.378740 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:32.391663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.391640 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:32.405820 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.405796 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:32.544064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.544025 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:32.639947 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.639917 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:32.640756 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.640739 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:32.869184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.869104 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:32.968096 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:32.968061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:34.664768 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:34.664727 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:37.408688 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:37.408649 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:37.704099 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:37.704019 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:42.709253 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.709168 2565 generic.go:356] "Generic (PLEG): container finished" podID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerID="75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac" exitCode=0 Jul 08 21:34:42.709627 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.709256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerDied","Data":"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac"} Jul 08 21:34:42.709672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.709646 2565 scope.go:117] "RemoveContainer" containerID="75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac" Jul 08 21:34:42.710808 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.710781 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:42.710878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.710864 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:42.710928 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:42.710912 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 21:34:43.548653 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:43.548615 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:43.567803 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:43.567766 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:43.681476 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:43.681450 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:43.717394 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:43.717359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerStarted","Data":"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c"} Jul 08 21:34:43.717814 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:43.717399 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" containerID="cri-o://4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c" gracePeriod=30 Jul 08 21:34:45.277089 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:45.277057 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:47.640803 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:47.640773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:48.986605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:48.986549 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:49.080344 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080319 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080359 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080460 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080438 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080475 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080566 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080542 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080567 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080564 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:34:49.080661 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080626 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:34:49.080820 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080695 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z5mw\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080820 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080713 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:34:49.080820 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080733 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token\") pod \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\" (UID: \"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6\") " Jul 08 21:34:49.080980 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.080887 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:34:49.081109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.081087 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istiod-ca-cert\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.081226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.081111 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.081226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.081125 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-credential-socket\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.081226 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.081138 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-workload-socket\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.081429 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.081327 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data" (OuterVolumeSpecName: "istio-data") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:34:49.082633 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.082607 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 21:34:49.082769 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.082747 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token" (OuterVolumeSpecName: "istio-token") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:34:49.082854 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.082787 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw" (OuterVolumeSpecName: "kube-api-access-5z5mw") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "kube-api-access-5z5mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:34:49.082854 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.082833 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" (UID: "024489bc-aac7-4cca-ab46-bc7ff9f9ecd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:34:49.181874 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.181848 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-data\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.181874 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.181873 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-envoy\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.182007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.181886 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-podinfo\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.182007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.181898 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z5mw\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-kube-api-access-5z5mw\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.182007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.181911 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6-istio-token\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:34:49.404814 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.404790 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:49.508821 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.508785 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:49.620677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.620648 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:49.716858 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.716772 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:49.755596 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.755570 2565 generic.go:356] "Generic (PLEG): container finished" podID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerID="4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c" exitCode=0 Jul 08 21:34:49.755718 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.755617 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" Jul 08 21:34:49.755718 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.755651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerDied","Data":"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c"} Jul 08 21:34:49.755718 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.755704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx" event={"ID":"024489bc-aac7-4cca-ab46-bc7ff9f9ecd6","Type":"ContainerDied","Data":"6dd89e07069a885a3fd6a541582b6390c82d3ca799e8dbb1b5ef1f49fe628b10"} Jul 08 21:34:49.755841 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.755729 2565 scope.go:117] "RemoveContainer" containerID="4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c" Jul 08 21:34:49.767733 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.767710 2565 scope.go:117] "RemoveContainer" containerID="75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac" Jul 08 21:34:49.777807 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.777786 2565 scope.go:117] "RemoveContainer" containerID="4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c" Jul 08 21:34:49.778087 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:34:49.778068 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c\": container with ID starting with 4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c not found: ID does not exist" containerID="4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c" Jul 08 21:34:49.778150 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.778094 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c"} err="failed to get container status \"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c\": rpc error: code = NotFound desc = could not find container \"4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c\": container with ID starting with 4bbac831c634aa0f757564656367a7b2d550338312eee4d481654912855f152c not found: ID does not exist" Jul 08 21:34:49.778150 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.778110 2565 scope.go:117] "RemoveContainer" containerID="75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac" Jul 08 21:34:49.778422 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:34:49.778404 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac\": container with ID starting with 75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac not found: ID does not exist" containerID="75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac" Jul 08 21:34:49.778473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.778427 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac"} err="failed to get container status \"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac\": rpc error: code = NotFound desc = could not find container \"75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac\": container with ID starting with 75f979820fdd8d1152a539b203bc4f2df0b9552f9054c6a7633b877d5a1b46ac not found: ID does not exist" Jul 08 21:34:49.786016 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.785996 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:49.789697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:49.789677 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-28a93421-openshift-default-5cc44b75-l5zvx"] Jul 08 21:34:50.708558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:50.708522 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:50.781251 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:50.781196 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:50.842610 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:50.842578 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:50.950489 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:50.950456 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" path="/var/lib/kubelet/pods/024489bc-aac7-4cca-ab46-bc7ff9f9ecd6/volumes" Jul 08 21:34:50.967095 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:50.967019 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:51.264346 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:51.264265 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:51.365587 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:51.365551 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:52.874058 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:52.874022 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:55.193043 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:55.193008 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:55.398199 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:55.398157 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:55.532230 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:55.532189 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:55.589276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:55.589209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:59.214832 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:34:59.214792 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:01.089755 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:01.089722 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:01.190666 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:01.190634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:01.583994 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:01.583958 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:01.676350 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:01.676317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:01.779397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:01.779373 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:03.260978 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:03.260945 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.113004 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.112966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.172481 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.172449 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.245505 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.245467 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.349615 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.349572 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.445345 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.445259 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:09.843384 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:09.843356 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:35.887883 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:35.887847 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.519805 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:41.519773 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.585475 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:41.585448 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.666092 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:41.666060 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.765998 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:41.765967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:48.305801 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:48.305769 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:48.361153 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:48.361120 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:57.737741 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:35:57.737633 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.517602 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:03.517562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.575588 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:03.575558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.678197 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:03.678167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.779605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:03.779577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:05.016500 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:05.016474 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:36:05.027718 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:05.027698 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:36:14.327522 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:14.327488 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:15.239845 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:15.239810 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:20.797954 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:20.797913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:20.903457 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:20.903424 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:20.997109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:20.997075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:21.097324 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:21.097228 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:22.050453 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:22.050420 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:22.151312 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:22.151283 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:22.256168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:22.256140 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:24.087167 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:24.087133 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:29.817811 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:29.817775 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:29.876468 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:29.876434 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:30.530793 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:30.530758 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:36.151333 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:36.151298 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:36.213052 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:36.213019 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:36.663605 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:36.663571 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:36.744982 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:36.744953 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:37.121279 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:37.121225 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.142757 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.142707 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.231286 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.231230 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.325022 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.324989 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.773850 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.773802 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.876448 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.876415 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:40.976225 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:40.976195 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:44.713366 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:44.713335 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:45.405754 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:45.405715 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:45.504392 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:45.504362 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:45.960854 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:45.960824 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:51.521825 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:51.521786 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:51.619634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:51.619597 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:51.924951 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:51.924916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:52.281906 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:52.281872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:52.386997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:52.386963 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:58.121909 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:58.121879 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:58.179089 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:58.179061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:58.407668 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:58.407584 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:58.767860 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:58.767789 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:58.864872 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:58.864835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:59.021770 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:36:59.021688 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.531001 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:04.530962 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.584444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:04.584412 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.739657 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:04.739630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.840121 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:04.840057 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.940378 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:04.940347 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:10.625824 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:10.625781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:10.726358 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:10.726317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:17.737537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:17.737503 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:18.109994 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:18.109964 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:18.574544 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:18.574510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:21.371131 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:21.371097 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:21.469562 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:21.469525 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:21.649037 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:21.648948 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:21.903287 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:21.903214 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:22.095595 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:22.095564 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:22.221200 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:22.221111 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:25.918903 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:25.918851 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:33.535834 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:33.535804 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:48.711940 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:48.711898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:48.811990 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:48.811954 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:53.813585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:53.813552 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:54.267350 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:54.267262 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:54.366709 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:54.366680 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:54.524718 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:54.524629 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:59.972580 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:37:59.972542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:05.051006 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:05.050974 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:41.616302 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:41.616270 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:43.439849 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:43.439806 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:47.576166 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:47.576136 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:47.643267 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:47.643208 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:47.882665 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:47.882588 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:49.228102 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:49.228061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:49.331717 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:49.331683 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:49.435898 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:49.435865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:50.256608 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:50.256556 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:50.355664 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:50.355632 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:50.454663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:50.454618 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:51.932901 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:51.932863 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:57.617981 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:57.617901 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:57.714629 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:57.714597 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:58.308545 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:38:58.308510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:04.369762 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:04.369726 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:04.492229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:04.492197 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:04.687620 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:04.687544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:15.216867 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:15.216832 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:15.578181 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:15.578148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:36.588630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:36.588592 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.080276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:43.080217 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.182540 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:43.182510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.493498 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:43.493428 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.821714 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:43.821680 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.895934 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:43.895899 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.113941 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.113887 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.275090 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.275031 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.388657 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.388586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.495395 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.495359 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.847829 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.847786 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:47.953328 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:47.953296 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:51.605294 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:51.605252 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:51.959520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:51.959442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:52.337919 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:52.337881 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:53.092352 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:53.092317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.020025 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.019976 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.100909 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.100872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.180310 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.180275 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.322916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.322883 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.733294 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.733198 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:55.832740 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:55.832706 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:59.549414 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:39:59.549370 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:07.469092 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:07.469058 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:08.576011 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:08.575977 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:08.676217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:08.676188 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:09.154599 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:09.154574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:17.649091 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:17.649059 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:17.745509 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:17.745480 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:39.450797 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:39.450750 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:58.654749 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:58.654721 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:58.706288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:40:58.706259 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:05.052591 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:05.052564 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:41:05.063900 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:05.063880 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:41:05.465030 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:05.464935 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:13.742790 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:13.742749 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:14.669534 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:14.669494 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:14.846123 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:14.846056 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:15.257208 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:15.257173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:15.358877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:15.358850 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:15.506173 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:15.506134 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:20.347719 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:20.347677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:56.943347 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:56.943259 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:57.248739 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:57.248655 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:57.750380 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:57.750335 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:58.141613 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:58.141582 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:58.244365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:58.244333 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:58.342549 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:58.342514 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:58.456760 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:41:58.456678 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:03.799637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:03.799601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:03.900217 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:03.900185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:04.148021 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:04.147918 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:04.718708 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:04.718672 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:05.055169 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:05.055124 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:05.437168 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:05.437084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:07.722577 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:07.722536 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:07.800997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:07.800958 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:07.876444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:07.876405 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:07.977537 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:07.977455 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:08.267420 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:08.267336 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:08.372705 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:08.372676 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:12.263310 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:12.263272 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:12.793277 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:12.793228 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:13.161923 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:13.161822 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:13.842701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:13.842666 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:15.777440 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:15.777401 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:15.859405 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:15.859363 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:15.923254 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:15.923211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:16.241088 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:16.241056 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:16.571959 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:16.571927 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:16.672676 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:16.672645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:20.325462 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:20.325426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:28.635096 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:28.635059 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:29.034270 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:29.034223 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:29.132767 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:29.132736 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:29.299417 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:29.299325 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:34.805319 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:34.805285 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:34.906193 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:34.906160 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:35.081929 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:35.081845 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:35.479852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:35.479754 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:35.926607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:35.926577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:36.021963 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:36.021939 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:36.182488 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:36.182404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:36.922924 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:36.922857 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:41.798163 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:41.798094 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:41.860802 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:41.860767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:42.147336 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:42.147220 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:42.701846 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:42.701817 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:43.016111 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:43.016034 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:43.352114 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:43.352072 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:45.917878 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:45.917832 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:46.001494 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:46.001463 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:46.062960 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:46.062928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:46.487520 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:46.487486 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:46.589312 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:46.589287 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:46.702313 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:46.702281 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:49.504094 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:49.504057 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:49.604185 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:49.604150 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:50.455513 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:50.455466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:50.806754 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:50.806721 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:51.134368 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:51.134266 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:51.323567 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:51.323527 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:54.000205 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:54.000172 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:54.067202 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:54.067164 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:54.131852 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:54.131822 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:54.668703 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:54.668665 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:54.771920 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:54.771885 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:58.553367 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:42:58.553327 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:07.195103 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:07.195061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:07.296028 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:07.295994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:15.598064 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:15.597997 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:15.699116 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:15.699078 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:16.557101 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:16.557068 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:42.046051 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:43:42.046017 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:28.563663 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:44:28.563626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:00.135298 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.135201 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:45:00.137621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.135889 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.137621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.135908 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.137621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.136037 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.137621 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.136052 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.139396 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.139375 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:45:00.139502 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.139482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:45:00.140106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.140088 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.140106 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.140106 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="024489bc-aac7-4cca-ab46-bc7ff9f9ecd6" containerName="istio-proxy" Jul 08 21:45:00.142192 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.142166 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4j6cj\"" Jul 08 21:45:00.144276 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.144257 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:45:00.144560 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.144540 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:45:00.148331 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.148309 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:45:00.148450 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.148398 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:45:00.148877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.148864 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:45:00.151548 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.151526 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:45:00.172039 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.172015 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hffg8\" (UniqueName: \"kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8\") pod \"maas-api-key-cleanup-29725785-vbwth\" (UID: \"770b6816-1bf2-4f1c-a27d-7db5a01445d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:45:00.273449 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.273418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwt76\" (UniqueName: \"kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76\") pod \"maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7\" (UID: \"da3018c3-49fa-4210-8371-5b6e2021e34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:45:00.273554 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.273510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffg8\" (UniqueName: \"kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8\") pod \"maas-api-key-cleanup-29725785-vbwth\" (UID: \"770b6816-1bf2-4f1c-a27d-7db5a01445d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:45:00.273615 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.273552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw6jh\" (UniqueName: \"kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh\") pod \"maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv\" (UID: \"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:45:00.281049 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.281030 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hffg8\" (UniqueName: \"kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8\") pod \"maas-api-key-cleanup-29725785-vbwth\" (UID: \"770b6816-1bf2-4f1c-a27d-7db5a01445d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:45:00.374377 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.374347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwt76\" (UniqueName: \"kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76\") pod \"maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7\" (UID: \"da3018c3-49fa-4210-8371-5b6e2021e34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:45:00.374473 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.374428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw6jh\" (UniqueName: \"kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh\") pod \"maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv\" (UID: \"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:45:00.382426 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.382404 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw6jh\" (UniqueName: \"kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh\") pod \"maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv\" (UID: \"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:45:00.382533 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.382484 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwt76\" (UniqueName: \"kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76\") pod \"maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7\" (UID: \"da3018c3-49fa-4210-8371-5b6e2021e34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:45:00.455322 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.455267 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:45:00.464189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.464171 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:45:00.471873 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.471851 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:45:00.616184 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.616151 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:45:00.618461 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:45:00.618413 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770b6816_1bf2_4f1c_a27d_7db5a01445d7.slice/crio-72191ef2d3e7383a4bccee3b5ae1b5b0f963259f1ca660238389fe080a72ce46 WatchSource:0}: Error finding container 72191ef2d3e7383a4bccee3b5ae1b5b0f963259f1ca660238389fe080a72ce46: Status 404 returned error can't find the container with id 72191ef2d3e7383a4bccee3b5ae1b5b0f963259f1ca660238389fe080a72ce46 Jul 08 21:45:00.852060 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.852036 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:45:00.856213 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:00.856193 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:45:00.931500 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:45:00.931465 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3018c3_49fa_4210_8371_5b6e2021e34a.slice/crio-29ee847978b0d63fd461b5cfd55ad513665f153906e16dbb46b6d37bec845c54 WatchSource:0}: Error finding container 29ee847978b0d63fd461b5cfd55ad513665f153906e16dbb46b6d37bec845c54: Status 404 returned error can't find the container with id 29ee847978b0d63fd461b5cfd55ad513665f153906e16dbb46b6d37bec845c54 Jul 08 21:45:00.932305 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:45:00.932187 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9cae18_8ed3_4575_a90a_8fd3f0256cb1.slice/crio-c39dbe254770adc97748f2bb42597d9bfdb37fc9997ed0297ebd677dbe2865e9 WatchSource:0}: Error finding container c39dbe254770adc97748f2bb42597d9bfdb37fc9997ed0297ebd677dbe2865e9: Status 404 returned error can't find the container with id c39dbe254770adc97748f2bb42597d9bfdb37fc9997ed0297ebd677dbe2865e9 Jul 08 21:45:01.094765 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:01.094736 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" event={"ID":"da3018c3-49fa-4210-8371-5b6e2021e34a","Type":"ContainerStarted","Data":"29ee847978b0d63fd461b5cfd55ad513665f153906e16dbb46b6d37bec845c54"} Jul 08 21:45:01.096153 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:01.096128 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" event={"ID":"770b6816-1bf2-4f1c-a27d-7db5a01445d7","Type":"ContainerStarted","Data":"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1"} Jul 08 21:45:01.096275 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:01.096169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" event={"ID":"770b6816-1bf2-4f1c-a27d-7db5a01445d7","Type":"ContainerStarted","Data":"72191ef2d3e7383a4bccee3b5ae1b5b0f963259f1ca660238389fe080a72ce46"} Jul 08 21:45:01.097291 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:01.097271 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" event={"ID":"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1","Type":"ContainerStarted","Data":"c39dbe254770adc97748f2bb42597d9bfdb37fc9997ed0297ebd677dbe2865e9"} Jul 08 21:45:01.112587 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:01.112546 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" podStartSLOduration=1.112532574 podStartE2EDuration="1.112532574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:01.110406591 +0000 UTC m=+2336.682151451" watchObservedRunningTime="2026-07-08 21:45:01.112532574 +0000 UTC m=+2336.684277433" Jul 08 21:45:02.104565 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:02.104530 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" event={"ID":"da3018c3-49fa-4210-8371-5b6e2021e34a","Type":"ContainerStarted","Data":"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09"} Jul 08 21:45:02.106049 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:02.106023 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" event={"ID":"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1","Type":"ContainerStarted","Data":"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73"} Jul 08 21:45:02.120796 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:02.120744 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" podStartSLOduration=2.12073057 podStartE2EDuration="2.12073057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:02.118838027 +0000 UTC m=+2337.690582898" watchObservedRunningTime="2026-07-08 21:45:02.12073057 +0000 UTC m=+2337.692475465" Jul 08 21:45:02.133624 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:02.133577 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" podStartSLOduration=2.133563639 podStartE2EDuration="2.133563639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:02.131957646 +0000 UTC m=+2337.703702507" watchObservedRunningTime="2026-07-08 21:45:02.133563639 +0000 UTC m=+2337.705308499" Jul 08 21:45:44.850788 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:44.850751 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:45.726633 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:45.726599 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:52.857403 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:45:52.857368 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:00.011016 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.010933 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:46:00.011601 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.011189 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" podUID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" containerName="cleanup" containerID="cri-o://b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1" gracePeriod=30 Jul 08 21:46:00.013531 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.013511 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:46:00.013712 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.013690 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" podUID="da3018c3-49fa-4210-8371-5b6e2021e34a" containerName="cleanup" containerID="cri-o://ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09" gracePeriod=30 Jul 08 21:46:00.017916 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.017890 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:46:00.018160 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:00.018134 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" podUID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" containerName="cleanup" containerID="cri-o://10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73" gracePeriod=30 Jul 08 21:46:05.095384 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:05.095354 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:46:05.106581 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:05.106559 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:46:30.230769 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.230743 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:46:30.254195 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.254173 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:46:30.258282 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.258261 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:46:30.321425 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.321404 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw6jh\" (UniqueName: \"kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh\") pod \"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1\" (UID: \"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1\") " Jul 08 21:46:30.321555 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.321502 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hffg8\" (UniqueName: \"kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8\") pod \"770b6816-1bf2-4f1c-a27d-7db5a01445d7\" (UID: \"770b6816-1bf2-4f1c-a27d-7db5a01445d7\") " Jul 08 21:46:30.323486 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.323453 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8" (OuterVolumeSpecName: "kube-api-access-hffg8") pod "770b6816-1bf2-4f1c-a27d-7db5a01445d7" (UID: "770b6816-1bf2-4f1c-a27d-7db5a01445d7"). InnerVolumeSpecName "kube-api-access-hffg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:46:30.323579 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.323453 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh" (OuterVolumeSpecName: "kube-api-access-vw6jh") pod "2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" (UID: "2b9cae18-8ed3-4575-a90a-8fd3f0256cb1"). InnerVolumeSpecName "kube-api-access-vw6jh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:46:30.422333 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.422311 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwt76\" (UniqueName: \"kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76\") pod \"da3018c3-49fa-4210-8371-5b6e2021e34a\" (UID: \"da3018c3-49fa-4210-8371-5b6e2021e34a\") " Jul 08 21:46:30.422585 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.422572 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw6jh\" (UniqueName: \"kubernetes.io/projected/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1-kube-api-access-vw6jh\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:46:30.422629 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.422587 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hffg8\" (UniqueName: \"kubernetes.io/projected/770b6816-1bf2-4f1c-a27d-7db5a01445d7-kube-api-access-hffg8\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:46:30.423997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.423978 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76" (OuterVolumeSpecName: "kube-api-access-xwt76") pod "da3018c3-49fa-4210-8371-5b6e2021e34a" (UID: "da3018c3-49fa-4210-8371-5b6e2021e34a"). InnerVolumeSpecName "kube-api-access-xwt76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:46:30.523627 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.523598 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwt76\" (UniqueName: \"kubernetes.io/projected/da3018c3-49fa-4210-8371-5b6e2021e34a-kube-api-access-xwt76\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\"" Jul 08 21:46:30.608234 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.608173 2565 generic.go:356] "Generic (PLEG): container finished" podID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" containerID="b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1" exitCode=137 Jul 08 21:46:30.608234 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.608217 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" Jul 08 21:46:30.608391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.608265 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" event={"ID":"770b6816-1bf2-4f1c-a27d-7db5a01445d7","Type":"ContainerDied","Data":"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1"} Jul 08 21:46:30.608391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.608311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth" event={"ID":"770b6816-1bf2-4f1c-a27d-7db5a01445d7","Type":"ContainerDied","Data":"72191ef2d3e7383a4bccee3b5ae1b5b0f963259f1ca660238389fe080a72ce46"} Jul 08 21:46:30.608391 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.608333 2565 scope.go:117] "RemoveContainer" containerID="b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1" Jul 08 21:46:30.609778 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.609753 2565 generic.go:356] "Generic (PLEG): container finished" podID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" containerID="10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73" exitCode=137 Jul 08 21:46:30.609877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.609787 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" Jul 08 21:46:30.609877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.609829 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" event={"ID":"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1","Type":"ContainerDied","Data":"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73"} Jul 08 21:46:30.609962 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.609874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv" event={"ID":"2b9cae18-8ed3-4575-a90a-8fd3f0256cb1","Type":"ContainerDied","Data":"c39dbe254770adc97748f2bb42597d9bfdb37fc9997ed0297ebd677dbe2865e9"} Jul 08 21:46:30.611547 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.611524 2565 generic.go:356] "Generic (PLEG): container finished" podID="da3018c3-49fa-4210-8371-5b6e2021e34a" containerID="ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09" exitCode=137 Jul 08 21:46:30.611640 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.611559 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" event={"ID":"da3018c3-49fa-4210-8371-5b6e2021e34a","Type":"ContainerDied","Data":"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09"} Jul 08 21:46:30.611640 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.611582 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" Jul 08 21:46:30.611640 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.611608 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7" event={"ID":"da3018c3-49fa-4210-8371-5b6e2021e34a","Type":"ContainerDied","Data":"29ee847978b0d63fd461b5cfd55ad513665f153906e16dbb46b6d37bec845c54"} Jul 08 21:46:30.619141 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.619123 2565 scope.go:117] "RemoveContainer" containerID="b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1" Jul 08 21:46:30.619519 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:46:30.619498 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1\": container with ID starting with b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1 not found: ID does not exist" containerID="b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1" Jul 08 21:46:30.619620 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.619525 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1"} err="failed to get container status \"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1\": rpc error: code = NotFound desc = could not find container \"b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1\": container with ID starting with b1d8a951c7a43369030161830cc8a0923c64c76f5a13b80030716e8332c6a0a1 not found: ID does not exist" Jul 08 21:46:30.619620 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.619546 2565 scope.go:117] "RemoveContainer" containerID="10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73" Jul 08 21:46:30.627387 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.627367 2565 scope.go:117] "RemoveContainer" containerID="10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73" Jul 08 21:46:30.627646 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:46:30.627628 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73\": container with ID starting with 10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73 not found: ID does not exist" containerID="10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73" Jul 08 21:46:30.627697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.627653 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73"} err="failed to get container status \"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73\": rpc error: code = NotFound desc = could not find container \"10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73\": container with ID starting with 10c18db437198be4a1be7485ea7e7fbd00fd34ebc3cd40ba590d5c4193faba73 not found: ID does not exist" Jul 08 21:46:30.627697 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.627668 2565 scope.go:117] "RemoveContainer" containerID="ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09" Jul 08 21:46:30.634471 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.634446 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:46:30.635997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.635980 2565 scope.go:117] "RemoveContainer" containerID="ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09" Jul 08 21:46:30.636314 ip-10-0-128-213 kubenswrapper[2565]: E0708 21:46:30.636288 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09\": container with ID starting with ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09 not found: ID does not exist" containerID="ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09" Jul 08 21:46:30.636403 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.636319 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09"} err="failed to get container status \"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09\": rpc error: code = NotFound desc = could not find container \"ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09\": container with ID starting with ef8678975ea7e34a202b8f96d10356c7684312557ce852ac0151ab9d5a6a5d09 not found: ID does not exist" Jul 08 21:46:30.636946 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.636929 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-vbwth"] Jul 08 21:46:30.646229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.646210 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:46:30.648538 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.648521 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6edaf8-29725785-sdzbv"] Jul 08 21:46:30.657303 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.657284 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:46:30.660544 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.660527 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fbb91c-29725785-qbhc7"] Jul 08 21:46:30.957529 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.957456 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" path="/var/lib/kubelet/pods/2b9cae18-8ed3-4575-a90a-8fd3f0256cb1/volumes" Jul 08 21:46:30.958031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.958013 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" path="/var/lib/kubelet/pods/770b6816-1bf2-4f1c-a27d-7db5a01445d7/volumes" Jul 08 21:46:30.958441 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:46:30.958427 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3018c3-49fa-4210-8371-5b6e2021e34a" path="/var/lib/kubelet/pods/da3018c3-49fa-4210-8371-5b6e2021e34a/volumes" Jul 08 21:47:02.017288 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:47:02.017232 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:47:50.172710 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:47:50.172669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:47:50.272801 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:47:50.272772 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:47:53.793154 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:47:53.793117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:47:54.724607 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:47:54.724576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:01.358908 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:01.358814 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:08.297969 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:08.297932 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:08.350690 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:08.350667 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:16.416790 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:16.416751 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:16.517771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:16.517736 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:19.824989 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:48:19.824958 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:06.582770 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:06.582726 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:15.182778 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:15.182740 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:15.234798 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:15.234761 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:17.944523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:17.944488 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:18.689528 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:18.689498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:26.159426 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:26.159345 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:43.498970 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:43.498933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:45.488409 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:45.488374 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:58.897975 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:58.897940 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:59.303818 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:59.303783 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:59.404254 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:59.404200 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:49:59.549062 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:49:59.549015 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:05.200955 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:05.200921 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:05.301360 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:05.301325 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:05.462114 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:05.462005 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:05.848818 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:05.848779 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:06.263885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:06.263805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:06.364381 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:06.364351 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:06.525067 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:06.524973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:12.378685 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:12.378647 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:12.465162 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:12.465121 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:12.660289 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:12.660188 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:18.294651 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:18.294621 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kvzf8_7d9c2233-9c8b-4608-89c4-5d0ca59539bd/manager/0.log" Jul 08 21:50:18.704836 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:18.704773 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-969j8_683e2f1e-0255-448c-bbd0-a0e2410f6317/server/0.log" Jul 08 21:50:18.839877 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:18.839838 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zfsgx_f218d1c2-bf24-4766-a4a0-073084d94417/manager/0.log" Jul 08 21:50:18.966870 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:18.966814 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94bc986b9-dcpp4_e315be0a-8872-40ce-b8c4-c0c1285a4bfa/copy-manifests/0.log" Jul 08 21:50:18.985594 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:18.985568 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94bc986b9-dcpp4_e315be0a-8872-40ce-b8c4-c0c1285a4bfa/manager/0.log" Jul 08 21:50:20.772544 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:20.772509 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-khvcx_1ef84a54-0db0-4d7b-969e-6e865da7350d/manager/0.log" Jul 08 21:50:20.897573 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:20.897544 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-99n96_4646a204-e883-4e2b-ba75-d3f45949af17/manager/0.log" Jul 08 21:50:21.014796 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:21.014769 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7w5x_2e06d0a8-7e51-4db1-87f6-078aaf947050/kuadrant-console-plugin/0.log" Jul 08 21:50:21.146894 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:21.146869 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k7cqx_601d6727-d641-4d2f-8dd9-525dec415fd3/registry-server/0.log" Jul 08 21:50:21.283397 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:21.283368 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gjnwc_4ed42850-f404-44d6-a4ae-5fb703b92fff/manager/0.log" Jul 08 21:50:21.541407 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:21.541385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nxghj_f04e6e8b-8e33-4c52-89a5-ceadbe32996b/manager/0.log" Jul 08 21:50:21.921065 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:21.920988 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49zf9g_f8150f62-9c40-4968-b6dc-9f97bc577b1e/istio-proxy/0.log" Jul 08 21:50:22.296651 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:22.296626 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7c4b48c4-zgwwt_8db80eb2-0065-455b-bc43-14efbcacf8d9/kube-auth-proxy/0.log" Jul 08 21:50:22.430444 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:22.430413 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/1.log" Jul 08 21:50:23.190305 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:23.190274 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gh6sw_f284c50a-46a0-45c4-a709-0477cd842442/main/0.log" Jul 08 21:50:23.315031 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:23.315006 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-fhczx_efd88a8f-916a-4a50-8516-da51c424487d/main/0.log" Jul 08 21:50:23.434159 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:23.434127 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-t4xtw_7a668954-773d-49bc-948d-0e219dc42eb6/main/0.log" Jul 08 21:50:28.371958 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:28.371922 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:28.468393 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:28.468358 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:31.178838 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:31.178806 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8cgls_6193c5e9-3127-4409-9460-90223187c853/global-pull-secret-syncer/0.log" Jul 08 21:50:31.271089 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:31.271060 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ckdxv_05078ba8-668e-4f03-88ec-75ba7ad49383/konnectivity-agent/0.log" Jul 08 21:50:31.366523 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:31.366494 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-213.ec2.internal_d32b06a52fe2c12e89c7b42a8cc8f0b1/haproxy/0.log" Jul 08 21:50:36.431377 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.431345 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-khvcx_1ef84a54-0db0-4d7b-969e-6e865da7350d/manager/0.log" Jul 08 21:50:36.481324 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.481293 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-99n96_4646a204-e883-4e2b-ba75-d3f45949af17/manager/0.log" Jul 08 21:50:36.507641 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.507619 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7w5x_2e06d0a8-7e51-4db1-87f6-078aaf947050/kuadrant-console-plugin/0.log" Jul 08 21:50:36.563859 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.563817 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k7cqx_601d6727-d641-4d2f-8dd9-525dec415fd3/registry-server/0.log" Jul 08 21:50:36.702161 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.702099 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gjnwc_4ed42850-f404-44d6-a4ae-5fb703b92fff/manager/0.log" Jul 08 21:50:36.836122 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:36.836097 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nxghj_f04e6e8b-8e33-4c52-89a5-ceadbe32996b/manager/0.log" Jul 08 21:50:38.616094 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.616069 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/alertmanager/0.log" Jul 08 21:50:38.639901 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.639878 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/config-reloader/0.log" Jul 08 21:50:38.659405 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.659377 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/kube-rbac-proxy-web/0.log" Jul 08 21:50:38.681493 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.681473 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/kube-rbac-proxy/0.log" Jul 08 21:50:38.703008 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.702987 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/kube-rbac-proxy-metric/0.log" Jul 08 21:50:38.735069 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.735052 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/prom-label-proxy/0.log" Jul 08 21:50:38.759231 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.759178 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b2c8d16-aa82-4462-b9f8-78176a1255bf/init-config-reloader/0.log" Jul 08 21:50:38.940277 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:38.940229 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-74f69594c9-k2jzc_c0149484-2084-4ab5-9ff4-e7f737e32494/metrics-server/0.log" Jul 08 21:50:39.001642 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.001620 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nr7_113e034e-b9c8-4d67-807f-4e32c414fa3b/node-exporter/0.log" Jul 08 21:50:39.024053 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.023999 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nr7_113e034e-b9c8-4d67-807f-4e32c414fa3b/kube-rbac-proxy/0.log" Jul 08 21:50:39.044935 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.044918 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22nr7_113e034e-b9c8-4d67-807f-4e32c414fa3b/init-textfile/0.log" Jul 08 21:50:39.313701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.313675 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/prometheus/0.log" Jul 08 21:50:39.331997 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.331973 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/config-reloader/0.log" Jul 08 21:50:39.351514 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.351497 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/thanos-sidecar/0.log" Jul 08 21:50:39.371688 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.371671 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/kube-rbac-proxy-web/0.log" Jul 08 21:50:39.391887 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.391852 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/kube-rbac-proxy/0.log" Jul 08 21:50:39.412613 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.412592 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/kube-rbac-proxy-thanos/0.log" Jul 08 21:50:39.431404 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.431384 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8fc5e2ea-ca74-47f8-a8b6-7df455e0b278/init-config-reloader/0.log" Jul 08 21:50:39.460652 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.460622 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-f4gb5_f6210c44-7749-42e2-9184-bb1d279b2a6e/prometheus-operator/0.log" Jul 08 21:50:39.477637 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.477620 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-f4gb5_f6210c44-7749-42e2-9184-bb1d279b2a6e/kube-rbac-proxy/0.log" Jul 08 21:50:39.500552 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.500534 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-x5sr9_28e67d46-c60c-4d74-902f-e755e059f45b/prometheus-operator-admission-webhook/0.log" Jul 08 21:50:39.529278 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.529258 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4786df7f-6lrns_732f43ac-236d-4bae-b9eb-304bda1cdc51/telemeter-client/0.log" Jul 08 21:50:39.553558 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.553537 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4786df7f-6lrns_732f43ac-236d-4bae-b9eb-304bda1cdc51/reload/0.log" Jul 08 21:50:39.573980 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:39.573930 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4786df7f-6lrns_732f43ac-236d-4bae-b9eb-304bda1cdc51/kube-rbac-proxy/0.log" Jul 08 21:50:40.888894 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:40.888866 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-898wv_549e2d5f-64f0-438b-9d66-ccd547074e9d/networking-console-plugin/0.log" Jul 08 21:50:41.522121 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.522056 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/1.log" Jul 08 21:50:41.532590 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.532561 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ztfgv_5bf1d832-d339-4c2e-a370-aa391743633c/console-operator/2.log" Jul 08 21:50:41.624138 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624111 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg"] Jul 08 21:50:41.624701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624682 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" containerName="cleanup" Jul 08 21:50:41.624701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624702 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" containerName="cleanup" Jul 08 21:50:41.624855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624714 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3018c3-49fa-4210-8371-5b6e2021e34a" containerName="cleanup" Jul 08 21:50:41.624855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624722 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3018c3-49fa-4210-8371-5b6e2021e34a" containerName="cleanup" Jul 08 21:50:41.624855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624739 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" containerName="cleanup" Jul 08 21:50:41.624855 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624747 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" containerName="cleanup" Jul 08 21:50:41.625051 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624874 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="770b6816-1bf2-4f1c-a27d-7db5a01445d7" containerName="cleanup" Jul 08 21:50:41.625051 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624888 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3018c3-49fa-4210-8371-5b6e2021e34a" containerName="cleanup" Jul 08 21:50:41.625051 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.624899 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b9cae18-8ed3-4575-a90a-8fd3f0256cb1" containerName="cleanup" Jul 08 21:50:41.628431 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.628412 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.631646 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.631627 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ntfkr\"/\"default-dockercfg-hzz45\"" Jul 08 21:50:41.631753 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.631670 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ntfkr\"/\"openshift-service-ca.crt\"" Jul 08 21:50:41.632673 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.632653 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ntfkr\"/\"kube-root-ca.crt\"" Jul 08 21:50:41.638363 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.638318 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg"] Jul 08 21:50:41.717451 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.717428 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-sys\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.717546 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.717474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjqx\" (UniqueName: \"kubernetes.io/projected/026497bb-9321-445b-96f6-0e705d72bb67-kube-api-access-2kjqx\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.717589 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.717542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-proc\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.717634 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.717602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-lib-modules\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.717672 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.717628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-podres\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.818771 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-sys\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.818879 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjqx\" (UniqueName: \"kubernetes.io/projected/026497bb-9321-445b-96f6-0e705d72bb67-kube-api-access-2kjqx\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.818879 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-proc\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.818879 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-lib-modules\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.819007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818893 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-sys\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.819007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-proc\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.819007 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.818898 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-podres\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.819143 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.819019 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-podres\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.819143 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.819047 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/026497bb-9321-445b-96f6-0e705d72bb67-lib-modules\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.826644 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.826620 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kjqx\" (UniqueName: \"kubernetes.io/projected/026497bb-9321-445b-96f6-0e705d72bb67-kube-api-access-2kjqx\") pod \"perf-node-gather-daemonset-67nwg\" (UID: \"026497bb-9321-445b-96f6-0e705d72bb67\") " pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:41.939229 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:41.939208 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:42.075676 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:42.075649 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg"] Jul 08 21:50:42.077515 ip-10-0-128-213 kubenswrapper[2565]: W0708 21:50:42.077491 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod026497bb_9321_445b_96f6_0e705d72bb67.slice/crio-820f02ef75b81872265cd6576e124d628c64b48356c2e1da62eca7dc1ef8b91e WatchSource:0}: Error finding container 820f02ef75b81872265cd6576e124d628c64b48356c2e1da62eca7dc1ef8b91e: Status 404 returned error can't find the container with id 820f02ef75b81872265cd6576e124d628c64b48356c2e1da62eca7dc1ef8b91e Jul 08 21:50:42.129744 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:42.129727 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77c66f5f9f-fsqz8_160c488d-b811-493f-b80b-3cc45dd79e12/console/0.log" Jul 08 21:50:42.172922 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:42.172896 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-ql4h7_4ec95395-3efe-40f4-87bd-72afc81c739b/download-server/0.log" Jul 08 21:50:42.761794 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:42.761768 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-j5fm2_b429128e-1987-4465-a48c-219eaf47bd53/volume-data-source-validator/0.log" Jul 08 21:50:43.003677 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.003647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" event={"ID":"026497bb-9321-445b-96f6-0e705d72bb67","Type":"ContainerStarted","Data":"bcb9ce39704421a8307b1e7199c6c871b9ead56d45ce7a684a314afc6587dc79"} Jul 08 21:50:43.004157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.003688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" event={"ID":"026497bb-9321-445b-96f6-0e705d72bb67","Type":"ContainerStarted","Data":"820f02ef75b81872265cd6576e124d628c64b48356c2e1da62eca7dc1ef8b91e"} Jul 08 21:50:43.004157 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.003757 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:43.023886 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.023817 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" podStartSLOduration=2.02380642 podStartE2EDuration="2.02380642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:50:43.022914988 +0000 UTC m=+2678.594659847" watchObservedRunningTime="2026-07-08 21:50:43.02380642 +0000 UTC m=+2678.595551278" Jul 08 21:50:43.846109 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.846079 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5926b_029c9231-733f-43bd-b92c-17b495c2adeb/dns/0.log" Jul 08 21:50:43.869189 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:43.869161 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5926b_029c9231-733f-43bd-b92c-17b495c2adeb/kube-rbac-proxy/0.log" Jul 08 21:50:44.090453 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:44.090421 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4ccx_a1c66415-3f68-4a2a-9137-f3c95eb1cd13/dns-node-resolver/0.log" Jul 08 21:50:44.695630 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:44.695591 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl9b5_56d4bc64-7000-4b7a-bbd5-ad884d2e7262/node-ca/0.log" Jul 08 21:50:45.639986 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:45.639957 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49zf9g_f8150f62-9c40-4968-b6dc-9f97bc577b1e/istio-proxy/0.log" Jul 08 21:50:45.794628 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:45.794593 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7c4b48c4-zgwwt_8db80eb2-0065-455b-bc43-14efbcacf8d9/kube-auth-proxy/0.log" Jul 08 21:50:45.827799 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:45.827777 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/0.log" Jul 08 21:50:45.881756 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:45.881731 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/1.log" Jul 08 21:50:46.737346 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:46.737316 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p578w_b787178f-18f3-4d58-888d-760a65129311/serve-healthcheck-canary/0.log" Jul 08 21:50:47.197178 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:47.197150 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-hrmnd_d940ecf2-11d4-4c5e-84de-87745c55deb4/insights-operator/1.log" Jul 08 21:50:47.197365 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:47.197154 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-hrmnd_d940ecf2-11d4-4c5e-84de-87745c55deb4/insights-operator/0.log" Jul 08 21:50:47.350368 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:47.350341 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r6lhx_e99c6906-ce61-4faa-ba3f-160952a55a2d/kube-rbac-proxy/0.log" Jul 08 21:50:47.369330 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:47.369307 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r6lhx_e99c6906-ce61-4faa-ba3f-160952a55a2d/exporter/0.log" Jul 08 21:50:47.390689 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:47.390664 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r6lhx_e99c6906-ce61-4faa-ba3f-160952a55a2d/extractor/0.log" Jul 08 21:50:49.019371 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:49.019337 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ntfkr/perf-node-gather-daemonset-67nwg" Jul 08 21:50:49.535097 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:49.535069 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kvzf8_7d9c2233-9c8b-4608-89c4-5d0ca59539bd/manager/0.log" Jul 08 21:50:49.911343 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:49.911264 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-969j8_683e2f1e-0255-448c-bbd0-a0e2410f6317/server/0.log" Jul 08 21:50:50.011980 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:50.011952 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zfsgx_f218d1c2-bf24-4766-a4a0-073084d94417/manager/0.log" Jul 08 21:50:50.130190 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:50.130159 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94bc986b9-dcpp4_e315be0a-8872-40ce-b8c4-c0c1285a4bfa/manager/0.log" Jul 08 21:50:50.146541 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:50.146518 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94bc986b9-dcpp4_e315be0a-8872-40ce-b8c4-c0c1285a4bfa/copy-manifests/0.log" Jul 08 21:50:51.650794 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:51.650766 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-x25sw_7e508b52-c8b3-4445-b142-62267ea72e8a/openshift-lws-operator/0.log" Jul 08 21:50:51.903555 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:51.903529 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49zf9g_f8150f62-9c40-4968-b6dc-9f97bc577b1e/istio-proxy/0.log" Jul 08 21:50:52.029766 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:52.029735 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7c4b48c4-zgwwt_8db80eb2-0065-455b-bc43-14efbcacf8d9/kube-auth-proxy/0.log" Jul 08 21:50:52.050539 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:52.050514 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/0.log" Jul 08 21:50:52.103617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:52.103588 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/1.log" Jul 08 21:50:53.357949 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:53.357920 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49zf9g_f8150f62-9c40-4968-b6dc-9f97bc577b1e/istio-proxy/0.log" Jul 08 21:50:53.489702 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:53.489681 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7c4b48c4-zgwwt_8db80eb2-0065-455b-bc43-14efbcacf8d9/kube-auth-proxy/0.log" Jul 08 21:50:53.514536 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:53.514500 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/0.log" Jul 08 21:50:53.576571 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:53.576541 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jp7fm_c1b1fbf7-8b5f-4696-9f78-aed53988711b/istio-proxy/1.log" Jul 08 21:50:56.592618 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:56.592534 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dpdqc_cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8/migrator/0.log" Jul 08 21:50:56.612469 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:56.612443 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dpdqc_cdaed41a-f9c3-43c1-8eff-6055a5c4dcc8/graceful-termination/0.log" Jul 08 21:50:56.957775 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:56.957698 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-v7959_6c0aa75c-8b9a-46d8-86e9-492244a01b55/kube-storage-version-migrator-operator/1.log" Jul 08 21:50:56.959639 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:56.959612 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-v7959_6c0aa75c-8b9a-46d8-86e9-492244a01b55/kube-storage-version-migrator-operator/0.log" Jul 08 21:50:58.030137 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.030111 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/kube-multus-additional-cni-plugins/0.log" Jul 08 21:50:58.049617 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.049595 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/egress-router-binary-copy/0.log" Jul 08 21:50:58.068988 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.068968 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/cni-plugins/0.log" Jul 08 21:50:58.089281 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.089265 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/bond-cni-plugin/0.log" Jul 08 21:50:58.108833 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.108808 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/routeoverride-cni/0.log" Jul 08 21:50:58.128594 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.128578 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/whereabouts-cni-bincopy/0.log" Jul 08 21:50:58.147885 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.147864 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9cb7q_60ecf469-6188-4015-9553-54f322b6bfe8/whereabouts-cni/0.log" Jul 08 21:50:58.497012 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.496939 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hqhtz_6a1ad78f-6098-41e9-b9d8-6483fb351125/kube-multus/0.log" Jul 08 21:50:58.610331 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.610297 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rz5lw_fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3/network-metrics-daemon/0.log" Jul 08 21:50:58.631701 ip-10-0-128-213 kubenswrapper[2565]: I0708 21:50:58.631675 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rz5lw_fd2c37d4-5fbd-4a6c-8e6e-6d3aa631c2c3/kube-rbac-proxy/0.log"