Jul 17 19:56:04.628961 ip-10-0-142-80 systemd[1]: Starting Kubernetes Kubelet... Jul 17 19:56:05.031494 ip-10-0-142-80 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 19:56:05.031494 ip-10-0-142-80 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 19:56:05.031494 ip-10-0-142-80 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 19:56:05.031494 ip-10-0-142-80 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 19:56:05.031494 ip-10-0-142-80 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 19:56:05.032303 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.031515 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 19:56:05.036781 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036760 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.036781 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036780 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.036781 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036785 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036792 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036796 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036801 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036805 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036810 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036813 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036817 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036821 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036825 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036828 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036833 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036845 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036852 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036859 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036866 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036871 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036876 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036881 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036887 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.037020 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036892 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036898 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036902 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036907 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036912 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036916 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036921 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036925 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036930 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036935 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036939 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036944 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036949 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036953 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036958 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036963 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036967 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036972 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036976 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036980 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.037726 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036985 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036990 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.036995 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037000 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037005 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037012 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037017 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037021 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037025 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037030 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037034 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037040 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037045 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037049 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037054 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037059 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037064 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037070 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037074 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.038425 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037081 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037088 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037093 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037101 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037105 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037110 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037114 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037120 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037125 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037129 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037133 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037138 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037143 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037147 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037153 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037157 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037162 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037167 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037171 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.039268 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037177 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037182 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037187 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037192 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037196 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037202 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037207 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037214 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037220 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037226 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037231 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037236 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037240 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037245 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037250 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037254 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037259 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037264 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.037269 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.039884 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039125 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039138 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039144 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039149 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039154 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039160 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039164 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039169 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039173 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039178 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039183 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039187 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039192 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039196 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039205 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039212 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039218 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039223 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039228 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.040363 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039236 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039242 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039247 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039251 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039268 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039273 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039277 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039282 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039286 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039293 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039299 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039305 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039310 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039315 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039321 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039326 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039331 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039336 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039341 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.041146 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039346 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039351 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039356 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039360 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039365 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039370 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039374 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039379 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039385 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039391 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039396 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039400 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039405 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039410 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039415 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039419 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039424 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039428 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039432 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039437 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.041965 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039442 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039446 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039451 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039455 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039460 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039465 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039469 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039475 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039480 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039485 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039489 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039494 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039498 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039537 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039555 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039560 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039563 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039567 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039570 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039574 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.042549 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039577 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039580 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039582 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039585 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039590 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039592 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039595 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039598 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039601 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039604 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039608 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039610 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039613 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039627 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039630 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039633 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039636 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039639 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039642 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039645 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.043104 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.039649 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041846 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041863 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041867 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041871 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041876 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041879 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041882 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041885 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041889 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041892 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041895 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041898 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041901 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041904 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041907 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041910 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041913 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041916 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041918 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.043668 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041921 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041924 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041927 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041930 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041932 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041935 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041938 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041942 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041944 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041947 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041950 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041953 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041955 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041962 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041966 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041969 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041972 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041975 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041978 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041981 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.044209 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041986 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041990 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041992 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041995 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.041998 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042001 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042004 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042007 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042009 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042012 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042015 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042017 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042020 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042023 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042025 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042029 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042032 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042034 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042037 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042040 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.044751 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042043 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042045 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042048 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042051 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042053 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042056 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042060 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042063 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042066 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042069 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042072 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042075 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042079 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042083 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042086 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042089 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042092 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042095 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042099 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042102 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.045295 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042105 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042108 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042110 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042115 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042119 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042122 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042125 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042128 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042130 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042133 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042136 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042139 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042141 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042144 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042147 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042149 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042152 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042156 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.045876 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042160 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042164 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042236 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042244 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042251 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042255 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042260 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042263 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042267 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042272 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042276 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042279 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042283 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042287 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042290 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042293 2578 flags.go:64] FLAG: --cgroup-root="" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042296 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042300 2578 flags.go:64] FLAG: --client-ca-file="" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042303 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042308 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042313 2578 flags.go:64] FLAG: --cluster-domain="" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042318 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042323 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 19:56:05.046359 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042329 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042334 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042339 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042342 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042346 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042349 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042353 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042356 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042359 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042363 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042366 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042371 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042374 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042377 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042380 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042383 2578 flags.go:64] FLAG: --enable-server="true" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042386 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042392 2578 flags.go:64] FLAG: --event-burst="100" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042395 2578 flags.go:64] FLAG: --event-qps="50" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042398 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042402 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042404 2578 flags.go:64] FLAG: --eviction-hard="" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042409 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042411 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042414 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 19:56:05.047026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042418 2578 flags.go:64] FLAG: --eviction-soft="" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042421 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042424 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042427 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042431 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042434 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042437 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042440 2578 flags.go:64] FLAG: --feature-gates="" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042444 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042448 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042451 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042454 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042457 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042461 2578 flags.go:64] FLAG: --help="false" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042464 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042467 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042470 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042473 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042477 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042480 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042484 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042487 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042490 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 19:56:05.047659 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042493 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042496 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042499 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042502 2578 flags.go:64] FLAG: --kube-reserved="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042505 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042508 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042511 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042514 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042517 2578 flags.go:64] FLAG: --lock-file="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042520 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042524 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042527 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042533 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042536 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042540 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042543 2578 flags.go:64] FLAG: --logging-format="text" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042545 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042549 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042552 2578 flags.go:64] FLAG: --manifest-url="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042555 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042561 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042564 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042572 2578 flags.go:64] FLAG: --max-pods="110" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042575 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042579 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 19:56:05.048319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042582 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042586 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042589 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042592 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042595 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042603 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042607 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042610 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042613 2578 flags.go:64] FLAG: --pod-cidr="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042616 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042622 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042625 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042628 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042631 2578 flags.go:64] FLAG: --port="10250" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042634 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042637 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0621e4a19a8c2cd37" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042641 2578 flags.go:64] FLAG: --qos-reserved="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042644 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042648 2578 flags.go:64] FLAG: --register-node="true" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042651 2578 flags.go:64] FLAG: --register-with-taints="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042655 2578 flags.go:64] FLAG: --registry-burst="10" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042658 2578 flags.go:64] FLAG: --registry-qps="5" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042662 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042665 2578 flags.go:64] FLAG: --reserved-memory="" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042669 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 19:56:05.049020 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042672 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042690 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042693 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042696 2578 flags.go:64] FLAG: --runonce="false" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042699 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042702 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042706 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042709 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042715 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042720 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042724 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042727 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042730 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042733 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042736 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042739 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042743 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042746 2578 flags.go:64] FLAG: --system-cgroups="" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042749 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042755 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042758 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042761 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042765 2578 flags.go:64] FLAG: --tls-min-version="" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042768 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042771 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 19:56:05.049676 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042774 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042778 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042780 2578 flags.go:64] FLAG: --v="2" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042785 2578 flags.go:64] FLAG: --version="false" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042789 2578 flags.go:64] FLAG: --vmodule="" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042795 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.042798 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042941 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042945 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042948 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042951 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042954 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042957 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042960 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042963 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042967 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042971 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042974 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042977 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042980 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042982 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.050369 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042985 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042988 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042991 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042994 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042996 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.042999 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043002 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043005 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043007 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043010 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043013 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043016 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043018 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043022 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043025 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043027 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043030 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043034 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043036 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043039 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.050956 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043042 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043044 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043047 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043050 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043053 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043055 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043062 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043067 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043070 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043073 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043075 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043078 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043081 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043084 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043087 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043091 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043095 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043098 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043101 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043104 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.051479 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043107 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043110 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043113 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043116 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043119 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043122 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043126 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043128 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043131 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043135 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043138 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043141 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043143 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043146 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043148 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043151 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043154 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043157 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043161 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043166 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.052018 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043169 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043171 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043174 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043177 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043180 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043182 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043185 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043188 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043191 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043193 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043196 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043199 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043202 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043206 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043209 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043212 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043216 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043218 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043221 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.052644 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043224 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043226 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043229 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043233 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043237 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.043240 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.043245 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.050489 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.050507 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.050522 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.050528 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050592 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050598 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050602 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.053323 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050605 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050609 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050611 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050614 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050618 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050621 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050624 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050627 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050630 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050633 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050636 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050638 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050641 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050644 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050647 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050650 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050654 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050659 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050661 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.053720 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050664 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050668 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050671 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050674 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050678 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050680 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050698 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050704 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050707 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050711 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050714 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050717 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050720 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050723 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050726 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050729 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050732 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050735 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050737 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050740 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.054220 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050743 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050747 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050751 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050755 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050757 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050760 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050763 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050766 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050769 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050771 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050774 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050777 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050780 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050782 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050785 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050788 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050792 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050794 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050799 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.054745 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050803 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050806 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050810 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050812 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050816 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050819 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050821 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050824 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050826 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050829 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050832 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050834 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050837 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050840 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050843 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050845 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050848 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050851 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050854 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050856 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.055248 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050859 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050862 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050865 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050867 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050870 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050873 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050876 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050878 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050881 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050884 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050887 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050889 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050893 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050896 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050899 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050902 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050905 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.055784 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.050908 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.050914 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051036 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051040 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051044 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051047 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051050 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051052 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051055 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051058 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051061 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051064 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051067 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051070 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051072 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 19:56:05.056243 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051075 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051078 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051081 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051083 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051086 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051089 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051092 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051094 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051097 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051100 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051103 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051106 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051109 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051113 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051116 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051119 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051122 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051126 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051129 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051132 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 19:56:05.056649 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051135 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051138 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051141 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051144 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051148 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051151 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051155 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051158 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051160 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051163 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051166 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051168 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051171 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051174 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051177 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051179 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051182 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051185 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051187 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 19:56:05.057191 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051190 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051192 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051195 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051198 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051200 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051204 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051208 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051212 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051216 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051219 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051222 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051225 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051228 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051231 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051234 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051237 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051240 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051242 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051245 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 19:56:05.057708 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051248 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051250 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051253 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051256 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051259 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051261 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051264 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051267 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051270 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051272 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051275 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051278 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051281 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051283 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051286 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051288 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051292 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051294 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051297 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051301 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 19:56:05.058206 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051304 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051306 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051309 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051311 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051314 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051317 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051319 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:05.051322 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.051327 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.051454 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.053783 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.055111 2578 server.go:1019] "Starting client certificate rotation" Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.055216 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 19:56:05.058737 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.055258 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 19:56:05.059080 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.057405 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 19:56:05.059080 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.058842 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 19:56:05.069567 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.069547 2578 log.go:25] "Validated CRI v1 runtime API" Jul 17 19:56:05.072421 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.072407 2578 log.go:25] "Validated CRI v1 image API" Jul 17 19:56:05.074048 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.074031 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 19:56:05.076391 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.076369 2578 fs.go:135] Filesystem UUIDs: map[6e8d338c-2ff2-4c29-af08-3317c81e4196:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a323dde6-0fc6-440e-aa23-8784bc779145:/dev/nvme0n1p3] Jul 17 19:56:05.076474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.076390 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 19:56:05.084535 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.084381 2578 manager.go:216] Machine: {Timestamp:2026-07-17 19:56:05.083003748 +0000 UTC m=+0.338235158 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101540 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2269afd1e1ab1987f4bf70d9a158a4 SystemUUID:ec2269af-d1e1-ab19-87f4-bf70d9a158a4 BootID:ecf067aa-1cef-4487-ae71-95d1a228fc62 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:bd:91:b4:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:bd:91:b4:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:e4:05:41:e9:7d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 19:56:05.084535 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.084519 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 19:56:05.084679 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.084625 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 19:56:05.086079 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.086056 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 19:56:05.087627 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.087598 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 19:56:05.087827 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.087633 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 19:56:05.088517 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.088506 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 19:56:05.088549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.088519 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 19:56:05.088549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.088527 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 19:56:05.089173 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.089162 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 19:56:05.090383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.090373 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 19:56:05.093204 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093192 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 17 19:56:05.093345 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093334 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 19:56:05.093382 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093370 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 17 19:56:05.093412 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093384 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 19:56:05.093442 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093411 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 19:56:05.093442 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093426 2578 kubelet.go:398] "Adding apiserver pod source" Jul 17 19:56:05.093442 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093438 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 19:56:05.093879 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093867 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 19:56:05.093919 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.093887 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 19:56:05.096361 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.096342 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 19:56:05.097878 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.097864 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 19:56:05.097924 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.097901 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 19:56:05.098587 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098575 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 19:56:05.098625 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098592 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 19:56:05.098625 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098599 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 19:56:05.098625 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098605 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 19:56:05.098625 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098614 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 19:56:05.098625 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098620 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098629 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098635 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098642 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098648 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098657 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 19:56:05.098791 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.098662 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 19:56:05.099486 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.099475 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 19:56:05.099486 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.099485 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 19:56:05.102887 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.102873 2578 server.go:1268] "Started kubelet" Jul 17 19:56:05.103018 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.102965 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 19:56:05.103081 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.103045 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 19:56:05.103184 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.103148 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 19:56:05.104024 ip-10-0-142-80 systemd[1]: Started Kubernetes Kubelet. Jul 17 19:56:05.104181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.104163 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 19:56:05.106126 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.106101 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 17 19:56:05.108204 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.108182 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 19:56:05.110460 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.110430 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 19:56:05.110678 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.110662 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 19:56:05.110839 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.110819 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 19:56:05.111080 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111054 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 19:56:05.111209 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111090 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 19:56:05.111209 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.111199 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.111812 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.110381 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-80.ec2.internal.18c32beadd3b65f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-80.ec2.internal,UID:ip-10-0-142-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-80.ec2.internal,},FirstTimestamp:2026-07-17 19:56:05.102847478 +0000 UTC m=+0.358078889,LastTimestamp:2026-07-17 19:56:05.102847478 +0000 UTC m=+0.358078889,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-80.ec2.internal,}" Jul 17 19:56:05.111909 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111895 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 19:56:05.111949 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111908 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 19:56:05.111949 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111923 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 19:56:05.111949 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111930 2578 factory.go:55] Registering systemd factory Jul 17 19:56:05.112143 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.111972 2578 factory.go:223] Registration of the systemd container factory successfully Jul 17 19:56:05.112143 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112050 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 17 19:56:05.112143 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112059 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 17 19:56:05.112242 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112199 2578 factory.go:153] Registering CRI-O factory Jul 17 19:56:05.112242 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112214 2578 factory.go:223] Registration of the crio container factory successfully Jul 17 19:56:05.112308 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112271 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 19:56:05.112308 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112296 2578 factory.go:103] Registering Raw factory Jul 17 19:56:05.112368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.112312 2578 manager.go:1196] Started watching for new ooms in manager Jul 17 19:56:05.113707 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.113177 2578 manager.go:349] Starting recovery of all containers Jul 17 19:56:05.118292 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.118152 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 19:56:05.118292 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.118265 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 19:56:05.126172 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.125945 2578 manager.go:354] Recovery completed Jul 17 19:56:05.130728 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.130713 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.133027 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.132999 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.133127 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133042 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.133127 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133058 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.133678 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133638 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 19:56:05.133678 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133670 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 19:56:05.133764 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133687 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 17 19:56:05.133764 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.133650 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.134517 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134504 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.134560 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134530 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.134560 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134542 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.134913 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134902 2578 policy_none.go:49] "None policy: Start" Jul 17 19:56:05.134960 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134917 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 19:56:05.134960 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.134928 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 19:56:05.135565 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.135492 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-80.ec2.internal.18c32beadf07e3c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-80.ec2.internal,UID:ip-10-0-142-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-80.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-80.ec2.internal,},FirstTimestamp:2026-07-17 19:56:05.133026241 +0000 UTC m=+0.388257652,LastTimestamp:2026-07-17 19:56:05.133026241 +0000 UTC m=+0.388257652,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-80.ec2.internal,}" Jul 17 19:56:05.135981 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.135968 2578 policy_none.go:47] "Start" Jul 17 19:56:05.147912 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.147840 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-80.ec2.internal.18c32beadf084126 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-80.ec2.internal,UID:ip-10-0-142-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-80.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-80.ec2.internal,},FirstTimestamp:2026-07-17 19:56:05.13305015 +0000 UTC m=+0.388281565,LastTimestamp:2026-07-17 19:56:05.13305015 +0000 UTC m=+0.388281565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-80.ec2.internal,}" Jul 17 19:56:05.156965 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.156944 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cpxf4" Jul 17 19:56:05.161305 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.161233 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-80.ec2.internal.18c32beadf087372 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-80.ec2.internal,UID:ip-10-0-142-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-80.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-80.ec2.internal,},FirstTimestamp:2026-07-17 19:56:05.133063026 +0000 UTC m=+0.388294437,LastTimestamp:2026-07-17 19:56:05.133063026 +0000 UTC m=+0.388294437,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-80.ec2.internal,}" Jul 17 19:56:05.164907 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.164890 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cpxf4" Jul 17 19:56:05.176363 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176348 2578 manager.go:337] "Starting Device Plugin manager" Jul 17 19:56:05.176441 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.176390 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 19:56:05.176441 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176404 2578 server.go:87] "Starting device plugin registration server" Jul 17 19:56:05.176714 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176699 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 19:56:05.176771 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176717 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 19:56:05.176845 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176829 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 19:56:05.176957 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176919 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 19:56:05.176957 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.176933 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 19:56:05.177532 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.177512 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 19:56:05.177621 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.177573 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.245603 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.245553 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 19:56:05.246966 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.246940 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 19:56:05.247106 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.246986 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 19:56:05.247106 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.247011 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 19:56:05.247106 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.247050 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 19:56:05.252256 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.252236 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 19:56:05.277253 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.277228 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.278225 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.278203 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.278336 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.278238 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.278336 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.278249 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.278336 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.278278 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.287813 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.287758 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.287813 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.287786 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-80.ec2.internal\": node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.317507 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.317484 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.348240 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.348211 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"] Jul 17 19:56:05.348346 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.348283 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.350082 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.350065 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.350152 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.350099 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.350152 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.350113 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.351637 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.351624 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.351762 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.351747 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.351814 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.351777 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.352660 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352640 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.352660 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352649 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.352660 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352667 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.352903 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352675 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.352903 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352691 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.352903 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.352679 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.353866 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.353849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.353966 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.353882 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 19:56:05.354655 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.354636 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory" Jul 17 19:56:05.354745 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.354665 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 19:56:05.354745 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.354682 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID" Jul 17 19:56:05.371365 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.371335 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-80.ec2.internal\" not found" node="ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.375784 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.375768 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-80.ec2.internal\" not found" node="ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.414871 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.414834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.414871 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.414870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.415095 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.414891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.417582 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.417556 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.515172 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.515172 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.515396 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515232 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.515396 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.515396 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.515396 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.515303 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.518493 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.518471 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.618747 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.618665 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.674542 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.674506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.679849 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:05.679829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:05.719580 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.719545 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.820536 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.820493 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:05.920706 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:05.920602 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:06.021450 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:06.021414 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:06.056219 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.056192 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 19:56:06.056837 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.056335 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 19:56:06.121864 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:06.121833 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found" Jul 17 19:56:06.140307 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.140282 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 19:56:06.169699 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.169654 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 19:51:05 +0000 UTC" deadline="2028-03-09 17:48:07.404651022 +0000 UTC" Jul 17 19:56:06.169699 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.169693 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14421h52m1.234962076s" Jul 17 19:56:06.211753 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.211671 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" Jul 17 19:56:06.224184 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.224165 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 19:56:06.225913 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.225898 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" Jul 17 19:56:06.244153 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.244135 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 19:56:06.328624 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.328599 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 19:56:06.357846 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:06.357816 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc4a6bf9a8830c582e31a32617937e6.slice/crio-39f7b6972f350e3a31fb7ce8532d8376793a558e948543a422bb6f230e0fec99 WatchSource:0}: Error finding container 39f7b6972f350e3a31fb7ce8532d8376793a558e948543a422bb6f230e0fec99: Status 404 returned error can't find the container with id 39f7b6972f350e3a31fb7ce8532d8376793a558e948543a422bb6f230e0fec99 Jul 17 19:56:06.364009 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.363989 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 19:56:06.369449 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:06.369427 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e0fc6a14943475efe90a14e95086ef.slice/crio-e41f19b8f9fbcbe0b91eeaaa74f00396fc8bd1329b91f73154d48fa7295ed36f WatchSource:0}: Error finding container e41f19b8f9fbcbe0b91eeaaa74f00396fc8bd1329b91f73154d48fa7295ed36f: Status 404 returned error can't find the container with id e41f19b8f9fbcbe0b91eeaaa74f00396fc8bd1329b91f73154d48fa7295ed36f Jul 17 19:56:06.716427 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.716401 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 19:56:06.978199 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:06.978115 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 19:56:07.095302 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.095269 2578 apiserver.go:52] "Watching apiserver" Jul 17 19:56:07.103097 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.103072 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 19:56:07.104239 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.104214 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-87pgj","openshift-multus/network-metrics-daemon-8gscn","openshift-network-diagnostics/network-check-target-srdd9","kube-system/konnectivity-agent-zfxzd","kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal","openshift-multus/multus-additional-cni-plugins-dz8sz","openshift-network-operator/iptables-alerter-684c9","openshift-ovn-kubernetes/ovnkube-node-5dzmx","kube-system/global-pull-secret-syncer-rqfrh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq","openshift-cluster-node-tuning-operator/tuned-8djmx","openshift-dns/node-resolver-jftrg","openshift-image-registry/node-ca-cq8td"] Jul 17 19:56:07.107033 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.107003 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.109963 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.109903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.110088 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110014 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6hggz\"" Jul 17 19:56:07.110152 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110112 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 19:56:07.110209 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110019 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 19:56:07.110334 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110307 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 19:56:07.110441 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110357 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.110526 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.110509 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.112010 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.111943 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 19:56:07.112010 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.111980 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.112199 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.112084 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 19:56:07.113106 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.113081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:07.113188 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.113173 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:07.115105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.113858 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.115105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.114093 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lfk7w\"" Jul 17 19:56:07.116445 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.116423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.118602 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.118566 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 19:56:07.118884 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.118851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m2dmj\"" Jul 17 19:56:07.118968 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.118850 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 19:56:07.118968 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.118920 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.120908 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.120891 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 19:56:07.121346 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.121285 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.121438 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.121351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.121438 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.121373 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rmdcj\"" Jul 17 19:56:07.121705 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.121666 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.123751 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.123751 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cnibin\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-device-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123862 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123876 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.123916 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-os-release\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.123921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-sys-fs\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pd9f4\"" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6jpt\" (UniqueName: \"kubernetes.io/projected/a22a6cc7-0a24-48ad-9943-70d15df02493-kube-api-access-f6jpt\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-socket-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-registration-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124203 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 19:56:07.124258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7vvj\" (UniqueName: \"kubernetes.io/projected/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-kube-api-access-x7vvj\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.124774 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124270 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.124970 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.124955 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 19:56:07.126955 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.126629 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.126955 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.126781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.127221 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.127198 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ntsh6\"" Jul 17 19:56:07.127331 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.127235 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.128938 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.128919 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.129211 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.129181 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.129211 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.129205 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pw6p2\"" Jul 17 19:56:07.129374 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.129269 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.129374 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.129344 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:07.131690 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.131671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.133903 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.133869 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 19:56:07.134228 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.134209 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.134315 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.134281 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:07.134315 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.134306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ms2s8\"" Jul 17 19:56:07.137085 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.136952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.139448 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139423 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 19:56:07.139557 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139545 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 19:56:07.139641 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139582 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 19:56:07.139641 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139620 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qqzn4\"" Jul 17 19:56:07.139787 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139699 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 19:56:07.139787 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.139704 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 19:56:07.140172 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.140156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 19:56:07.203754 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.203720 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cnvwg" Jul 17 19:56:07.213473 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.213447 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 19:56:07.216775 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.216746 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cnvwg" Jul 17 19:56:07.224582 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-ovn\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.224724 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-log-socket\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.224724 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.224724 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.224724 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224706 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-run\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-sys\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88x9\" (UniqueName: \"kubernetes.io/projected/67eb9374-9fb7-4b2f-8b35-eee4d7649140-kube-api-access-h88x9\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-env-overrides\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-agent-certs\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224897 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:07.224964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-kubernetes\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.224996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67eb9374-9fb7-4b2f-8b35-eee4d7649140-tmp-dir\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-daemon-config\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-multus-certs\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-systemd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-tmp\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.225312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpq79\" (UniqueName: \"kubernetes.io/projected/a0f3d694-caca-4f28-b36a-0d7e36afb234-kube-api-access-xpq79\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-kubelet\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-slash\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-netns\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-modprobe-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-var-lib-kubelet\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-os-release\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-etc-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457n2\" (UniqueName: \"kubernetes.io/projected/47f1e417-214a-49d4-aca6-7b0bb1fc0627-kube-api-access-457n2\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225674 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cnibin\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-device-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.225763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225769 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cnibin\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-conf\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225793 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-device-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225819 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-bin\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovn-node-metrics-cert\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjvv\" (UniqueName: \"kubernetes.io/projected/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-kube-api-access-4mjvv\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6jpt\" (UniqueName: \"kubernetes.io/projected/a22a6cc7-0a24-48ad-9943-70d15df02493-kube-api-access-f6jpt\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-cni-binary-copy\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.225987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-var-lib-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-config\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-socket-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226096 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226120 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-lib-modules\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-etc-kubernetes\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226175 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-socket-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.226616 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-kubelet\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-tuned\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-system-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2g8t\" (UniqueName: \"kubernetes.io/projected/002b2b02-4cf2-49f7-8ab6-cac22026c906-kube-api-access-n2g8t\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226381 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-systemd-units\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-node-log\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zck8t\" (UniqueName: \"kubernetes.io/projected/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-kube-api-access-zck8t\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67eb9374-9fb7-4b2f-8b35-eee4d7649140-hosts-file\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-netd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-os-release\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysconfig\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-hostroot\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.227474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-script-lib\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-konnectivity-ca\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-host-slash\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-registration-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-systemd\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-host\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5rz8\" (UniqueName: \"kubernetes.io/projected/1d6ecb91-80ae-4221-9797-50fa1be7eda1-kube-api-access-x5rz8\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f3d694-caca-4f28-b36a-0d7e36afb234-host\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-cnibin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.226989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-dbus\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7vvj\" (UniqueName: \"kubernetes.io/projected/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-kube-api-access-x7vvj\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-registration-dir\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-socket-dir-parent\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227116 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-netns\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-os-release\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-multus\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.228344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-conf-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227304 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-iptables-alerter-script\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-sys-fs\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227394 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a22a6cc7-0a24-48ad-9943-70d15df02493-sys-fs\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f3d694-caca-4f28-b36a-0d7e36afb234-serviceca\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-bin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227498 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-kubelet-config\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.229232 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.227556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-k8s-cni-cncf-io\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.235266 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.235238 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 19:56:07.236656 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.236286 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:07.236656 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.236311 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:07.236656 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.236325 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:07.236656 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.236398 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:07.736367534 +0000 UTC m=+2.991598939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:07.239720 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.239637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6jpt\" (UniqueName: \"kubernetes.io/projected/a22a6cc7-0a24-48ad-9943-70d15df02493-kube-api-access-f6jpt\") pod \"aws-ebs-csi-driver-node-hrxzq\" (UID: \"a22a6cc7-0a24-48ad-9943-70d15df02493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.239720 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.239648 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7vvj\" (UniqueName: \"kubernetes.io/projected/a69bc2e5-8958-461f-a7e9-dca1c0c070c5-kube-api-access-x7vvj\") pod \"multus-additional-cni-plugins-dz8sz\" (UID: \"a69bc2e5-8958-461f-a7e9-dca1c0c070c5\") " pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.255122 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.255063 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerStarted","Data":"e41f19b8f9fbcbe0b91eeaaa74f00396fc8bd1329b91f73154d48fa7295ed36f"} Jul 17 19:56:07.257294 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.257262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" event={"ID":"ddc4a6bf9a8830c582e31a32617937e6","Type":"ContainerStarted","Data":"39f7b6972f350e3a31fb7ce8532d8376793a558e948543a422bb6f230e0fec99"} Jul 17 19:56:07.328560 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-kubelet-config\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-k8s-cni-cncf-io\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-ovn\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328611 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-kubelet-config\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-log-socket\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-k8s-cni-cncf-io\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-run\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-sys\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-ovn\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88x9\" (UniqueName: \"kubernetes.io/projected/67eb9374-9fb7-4b2f-8b35-eee4d7649140-kube-api-access-h88x9\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.328770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328746 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-log-socket\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-env-overrides\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-agent-certs\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328848 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-sys\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-kubernetes\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67eb9374-9fb7-4b2f-8b35-eee4d7649140-tmp-dir\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-daemon-config\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-multus-certs\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-systemd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-tmp\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-kubernetes\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpq79\" (UniqueName: \"kubernetes.io/projected/a0f3d694-caca-4f28-b36a-0d7e36afb234-kube-api-access-xpq79\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-kubelet\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-slash\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-netns\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329213 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-modprobe-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.329278 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-var-lib-kubelet\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-os-release\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-etc-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457n2\" (UniqueName: \"kubernetes.io/projected/47f1e417-214a-49d4-aca6-7b0bb1fc0627-kube-api-access-457n2\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-conf\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-bin\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329367 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-env-overrides\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovn-node-metrics-cert\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjvv\" (UniqueName: \"kubernetes.io/projected/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-kube-api-access-4mjvv\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-cni-binary-copy\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-var-lib-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-config\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-lib-modules\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329503 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67eb9374-9fb7-4b2f-8b35-eee4d7649140-tmp-dir\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-etc-kubernetes\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.328807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-run\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-os-release\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.330215 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329639 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-etc-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329770 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-multus-certs\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-run-systemd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-bin\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329978 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-conf\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330091 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-daemon-config\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-lib-modules\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330146 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-etc-kubernetes\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330179 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-netns\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330212 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-kubelet\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330252 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-slash\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-kubelet\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-tuned\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-config\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.329507 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-modprobe-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.331177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-system-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2g8t\" (UniqueName: \"kubernetes.io/projected/002b2b02-4cf2-49f7-8ab6-cac22026c906-kube-api-access-n2g8t\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-var-lib-openvswitch\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330521 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-systemd-units\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.330554 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.330620 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:07.830599718 +0000 UTC m=+3.085831118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-systemd-units\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-system-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-node-log\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b2b02-4cf2-49f7-8ab6-cac22026c906-cni-binary-copy\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330883 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-node-log\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330918 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-kubelet\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.330985 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-cni-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zck8t\" (UniqueName: \"kubernetes.io/projected/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-kube-api-access-zck8t\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67eb9374-9fb7-4b2f-8b35-eee4d7649140-hosts-file\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-netd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysconfig\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.332335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-hostroot\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-script-lib\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331172 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67eb9374-9fb7-4b2f-8b35-eee4d7649140-hosts-file\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-konnectivity-ca\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331217 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-host-slash\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-hostroot\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-systemd\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-cni-netd\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-host\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331307 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysconfig\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-host\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-host-slash\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-systemd\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-var-lib-kubelet\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-konnectivity-ca\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5rz8\" (UniqueName: \"kubernetes.io/projected/1d6ecb91-80ae-4221-9797-50fa1be7eda1-kube-api-access-x5rz8\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f3d694-caca-4f28-b36a-0d7e36afb234-host\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.333245 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.331995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-cnibin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-dbus\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-socket-dir-parent\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f3d694-caca-4f28-b36a-0d7e36afb234-host\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-netns\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-multus\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-conf-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-iptables-alerter-script\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f3d694-caca-4f28-b36a-0d7e36afb234-serviceca\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-bin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovnkube-script-lib\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332334 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-multus\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-var-lib-cni-bin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-cnibin\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-conf-dir\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/908d7010-dec8-4574-9ce4-29a5d5327bec-dbus\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.334236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332591 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-multus-socket-dir-parent\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.335347 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332632 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b2b02-4cf2-49f7-8ab6-cac22026c906-host-run-netns\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.335347 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332699 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47f1e417-214a-49d4-aca6-7b0bb1fc0627-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.335347 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.332807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-sysctl-d\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.335347 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.334458 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:07.335347 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.334534 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:07.834515366 +0000 UTC m=+3.089746764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.335716 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-iptables-alerter-script\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.335890 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47f1e417-214a-49d4-aca6-7b0bb1fc0627-ovn-node-metrics-cert\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.335983 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f3d694-caca-4f28-b36a-0d7e36afb234-serviceca\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.336038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3889-3c38-4343-a98f-0b5326c1a9f4-agent-certs\") pod \"konnectivity-agent-zfxzd\" (UID: \"8fbb3889-3c38-4343-a98f-0b5326c1a9f4\") " pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.336241 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-etc-tuned\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.336289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.336273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-tmp\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.341296 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.341274 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88x9\" (UniqueName: \"kubernetes.io/projected/67eb9374-9fb7-4b2f-8b35-eee4d7649140-kube-api-access-h88x9\") pod \"node-resolver-jftrg\" (UID: \"67eb9374-9fb7-4b2f-8b35-eee4d7649140\") " pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.341656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.341627 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpq79\" (UniqueName: \"kubernetes.io/projected/a0f3d694-caca-4f28-b36a-0d7e36afb234-kube-api-access-xpq79\") pod \"node-ca-cq8td\" (UID: \"a0f3d694-caca-4f28-b36a-0d7e36afb234\") " pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.342122 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.342098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5rz8\" (UniqueName: \"kubernetes.io/projected/1d6ecb91-80ae-4221-9797-50fa1be7eda1-kube-api-access-x5rz8\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.344439 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.344140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zck8t\" (UniqueName: \"kubernetes.io/projected/3242d8ab-8ab7-4544-99e3-092ac4b7eef6-kube-api-access-zck8t\") pod \"tuned-8djmx\" (UID: \"3242d8ab-8ab7-4544-99e3-092ac4b7eef6\") " pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.344702 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.344583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjvv\" (UniqueName: \"kubernetes.io/projected/ca287ec2-874f-42af-8a4b-d42a1a1dce8c-kube-api-access-4mjvv\") pod \"iptables-alerter-684c9\" (UID: \"ca287ec2-874f-42af-8a4b-d42a1a1dce8c\") " pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.344702 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.344647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-457n2\" (UniqueName: \"kubernetes.io/projected/47f1e417-214a-49d4-aca6-7b0bb1fc0627-kube-api-access-457n2\") pod \"ovnkube-node-5dzmx\" (UID: \"47f1e417-214a-49d4-aca6-7b0bb1fc0627\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.346023 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.345996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2g8t\" (UniqueName: \"kubernetes.io/projected/002b2b02-4cf2-49f7-8ab6-cac22026c906-kube-api-access-n2g8t\") pod \"multus-87pgj\" (UID: \"002b2b02-4cf2-49f7-8ab6-cac22026c906\") " pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.421505 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.421464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" Jul 17 19:56:07.426157 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.426130 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" Jul 17 19:56:07.434389 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.434365 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:07.435251 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.435221 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69bc2e5_8958_461f_a7e9_dca1c0c070c5.slice/crio-cb19efdbd62a844dd46040b498f25033c48211c748fb42cb68c71fac934c0ca9 WatchSource:0}: Error finding container cb19efdbd62a844dd46040b498f25033c48211c748fb42cb68c71fac934c0ca9: Status 404 returned error can't find the container with id cb19efdbd62a844dd46040b498f25033c48211c748fb42cb68c71fac934c0ca9 Jul 17 19:56:07.435948 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.435909 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22a6cc7_0a24_48ad_9943_70d15df02493.slice/crio-b11bf435589b0713c48e4e5fecf02cc65deb2c80e0a1bc52ef5736e9fb3f29ad WatchSource:0}: Error finding container b11bf435589b0713c48e4e5fecf02cc65deb2c80e0a1bc52ef5736e9fb3f29ad: Status 404 returned error can't find the container with id b11bf435589b0713c48e4e5fecf02cc65deb2c80e0a1bc52ef5736e9fb3f29ad Jul 17 19:56:07.440758 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.440733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cq8td" Jul 17 19:56:07.446388 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.446362 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbb3889_3c38_4343_a98f_0b5326c1a9f4.slice/crio-b0803b07ec3c9f82870c0105adc7e1f0fab8185c879d53ab02fd47b6e3741fee WatchSource:0}: Error finding container b0803b07ec3c9f82870c0105adc7e1f0fab8185c879d53ab02fd47b6e3741fee: Status 404 returned error can't find the container with id b0803b07ec3c9f82870c0105adc7e1f0fab8185c879d53ab02fd47b6e3741fee Jul 17 19:56:07.447703 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.447679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-684c9" Jul 17 19:56:07.453139 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.453102 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f3d694_caca_4f28_b36a_0d7e36afb234.slice/crio-1eb3655acc622db97faed9404f46783d70eaab21bdd69e9cf7a0a5aea503a9b1 WatchSource:0}: Error finding container 1eb3655acc622db97faed9404f46783d70eaab21bdd69e9cf7a0a5aea503a9b1: Status 404 returned error can't find the container with id 1eb3655acc622db97faed9404f46783d70eaab21bdd69e9cf7a0a5aea503a9b1 Jul 17 19:56:07.455647 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.455620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8djmx" Jul 17 19:56:07.459839 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.459714 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca287ec2_874f_42af_8a4b_d42a1a1dce8c.slice/crio-7437a015d8ebf2014d51204bda848dafb668c2aa32d4c52b17bf81d1f0b39246 WatchSource:0}: Error finding container 7437a015d8ebf2014d51204bda848dafb668c2aa32d4c52b17bf81d1f0b39246: Status 404 returned error can't find the container with id 7437a015d8ebf2014d51204bda848dafb668c2aa32d4c52b17bf81d1f0b39246 Jul 17 19:56:07.465800 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.465776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jftrg" Jul 17 19:56:07.467036 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.467010 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3242d8ab_8ab7_4544_99e3_092ac4b7eef6.slice/crio-18ab8d840b666027007a6f35c9bb361b31e575d11492f70199b5729875bf4dd5 WatchSource:0}: Error finding container 18ab8d840b666027007a6f35c9bb361b31e575d11492f70199b5729875bf4dd5: Status 404 returned error can't find the container with id 18ab8d840b666027007a6f35c9bb361b31e575d11492f70199b5729875bf4dd5 Jul 17 19:56:07.476191 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.476164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-87pgj" Jul 17 19:56:07.477703 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.477657 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67eb9374_9fb7_4b2f_8b35_eee4d7649140.slice/crio-d294884d13f60246088d040a79bc30a0129b8c359750810928faaeb8ff402d30 WatchSource:0}: Error finding container d294884d13f60246088d040a79bc30a0129b8c359750810928faaeb8ff402d30: Status 404 returned error can't find the container with id d294884d13f60246088d040a79bc30a0129b8c359750810928faaeb8ff402d30 Jul 17 19:56:07.483753 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.483728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:07.489062 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:07.489036 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002b2b02_4cf2_49f7_8ab6_cac22026c906.slice/crio-6341c471a4604f12244c57d23647f88899e0ee00738d30378196df906d9f3b79 WatchSource:0}: Error finding container 6341c471a4604f12244c57d23647f88899e0ee00738d30378196df906d9f3b79: Status 404 returned error can't find the container with id 6341c471a4604f12244c57d23647f88899e0ee00738d30378196df906d9f3b79 Jul 17 19:56:07.837101 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.836995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:07.837101 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.837064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:07.837112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837194 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837256 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837281 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837297 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837257 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:07.837342 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837281 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:08.837258327 +0000 UTC m=+4.092489724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:07.837789 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837374 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:08.837353446 +0000 UTC m=+4.092584848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:07.837789 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:07.837393 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:08.837382746 +0000 UTC m=+4.092614150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:08.220784 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.220567 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 19:51:07 +0000 UTC" deadline="2028-03-07 17:37:32.520520704 +0000 UTC" Jul 17 19:56:08.220784 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.220607 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14373h41m24.299918608s" Jul 17 19:56:08.247604 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.247506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:08.247793 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.247697 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:08.261772 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.261733 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-87pgj" event={"ID":"002b2b02-4cf2-49f7-8ab6-cac22026c906","Type":"ContainerStarted","Data":"6341c471a4604f12244c57d23647f88899e0ee00738d30378196df906d9f3b79"} Jul 17 19:56:08.263741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.263698 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cq8td" event={"ID":"a0f3d694-caca-4f28-b36a-0d7e36afb234","Type":"ContainerStarted","Data":"1eb3655acc622db97faed9404f46783d70eaab21bdd69e9cf7a0a5aea503a9b1"} Jul 17 19:56:08.265235 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.265209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" event={"ID":"a22a6cc7-0a24-48ad-9943-70d15df02493","Type":"ContainerStarted","Data":"b11bf435589b0713c48e4e5fecf02cc65deb2c80e0a1bc52ef5736e9fb3f29ad"} Jul 17 19:56:08.266866 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.266841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jftrg" event={"ID":"67eb9374-9fb7-4b2f-8b35-eee4d7649140","Type":"ContainerStarted","Data":"d294884d13f60246088d040a79bc30a0129b8c359750810928faaeb8ff402d30"} Jul 17 19:56:08.268741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.268716 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8djmx" event={"ID":"3242d8ab-8ab7-4544-99e3-092ac4b7eef6","Type":"ContainerStarted","Data":"18ab8d840b666027007a6f35c9bb361b31e575d11492f70199b5729875bf4dd5"} Jul 17 19:56:08.270577 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.270550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-684c9" event={"ID":"ca287ec2-874f-42af-8a4b-d42a1a1dce8c","Type":"ContainerStarted","Data":"7437a015d8ebf2014d51204bda848dafb668c2aa32d4c52b17bf81d1f0b39246"} Jul 17 19:56:08.272297 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.272269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfxzd" event={"ID":"8fbb3889-3c38-4343-a98f-0b5326c1a9f4","Type":"ContainerStarted","Data":"b0803b07ec3c9f82870c0105adc7e1f0fab8185c879d53ab02fd47b6e3741fee"} Jul 17 19:56:08.273779 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.273714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerStarted","Data":"cb19efdbd62a844dd46040b498f25033c48211c748fb42cb68c71fac934c0ca9"} Jul 17 19:56:08.844285 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.844240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:08.844499 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.844319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:08.844499 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:08.844373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:08.844628 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.844521 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:08.844628 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.844579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:10.84456035 +0000 UTC m=+6.099791746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:08.845042 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.844895 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:08.845042 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.844971 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:10.844948328 +0000 UTC m=+6.100179732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:08.845042 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.845020 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:08.845042 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.845037 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:08.845318 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.845052 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:08.845318 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:08.845114 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:10.845087636 +0000 UTC m=+6.100319049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:09.049925 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:09.049649 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f1e417_214a_49d4_aca6_7b0bb1fc0627.slice/crio-9aa70dfd7d3878e58bfabe091bedc108ff8636d66245cc7ee3a9febe965cd2bb WatchSource:0}: Error finding container 9aa70dfd7d3878e58bfabe091bedc108ff8636d66245cc7ee3a9febe965cd2bb: Status 404 returned error can't find the container with id 9aa70dfd7d3878e58bfabe091bedc108ff8636d66245cc7ee3a9febe965cd2bb Jul 17 19:56:09.221093 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:09.221045 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 19:51:07 +0000 UTC" deadline="2028-03-10 01:57:57.480785663 +0000 UTC" Jul 17 19:56:09.221093 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:09.221093 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14430h1m48.259699196s" Jul 17 19:56:09.251344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:09.250182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:09.251344 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:09.250323 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:09.251344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:09.250752 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:09.251344 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:09.250928 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:09.292016 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:09.291969 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"9aa70dfd7d3878e58bfabe091bedc108ff8636d66245cc7ee3a9febe965cd2bb"} Jul 17 19:56:10.247440 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:10.247354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:10.247939 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.247492 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:10.864629 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:10.864574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:10.864809 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:10.864656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:10.864809 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:10.864709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:10.864944 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.864868 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:10.864944 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.864890 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:10.864944 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.864902 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:10.865081 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.864957 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:14.864940288 +0000 UTC m=+10.120171687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:10.865364 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.865344 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:10.865447 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.865399 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:14.865385179 +0000 UTC m=+10.120616577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:10.865514 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.865456 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:10.865514 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:10.865487 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:14.865476183 +0000 UTC m=+10.120707579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:11.248355 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:11.248322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:11.248819 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:11.248464 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:11.248819 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:11.248509 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:11.248819 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:11.248688 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:12.247987 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:12.247951 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:12.248162 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:12.248087 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:13.249236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:13.248723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:13.249236 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:13.248866 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:13.249236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:13.248906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:13.249236 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:13.248996 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:14.247504 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:14.247458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:14.247687 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.247590 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:14.897515 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:14.897473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:14.897537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:14.897584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897671 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897714 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897731 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897745 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:22.897729848 +0000 UTC m=+18.152961243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897748 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897761 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897776 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:22.897758599 +0000 UTC m=+18.152989996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:14.897968 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:14.897804 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:22.897790955 +0000 UTC m=+18.153022352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:15.248475 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:15.248392 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:15.248640 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:15.248510 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:15.248640 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:15.248555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:15.248771 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:15.248676 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:16.247633 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:16.247584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:16.248107 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:16.247738 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:17.247803 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:17.247710 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:17.247803 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:17.247756 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:17.248328 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:17.247859 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:17.248328 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:17.248298 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:18.248187 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:18.248098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:18.248727 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:18.248249 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:18.315566 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:18.315528 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" event={"ID":"ddc4a6bf9a8830c582e31a32617937e6","Type":"ContainerStarted","Data":"1b4b1727c4c5f9f17e2c31229c7b5d775ad45988590d71a3c219aeecc057084a"} Jul 17 19:56:18.331777 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:18.331713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" podStartSLOduration=12.331691599 podStartE2EDuration="12.331691599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:18.331213881 +0000 UTC m=+13.586445299" watchObservedRunningTime="2026-07-17 19:56:18.331691599 +0000 UTC m=+13.586923014" Jul 17 19:56:19.247914 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:19.247879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:19.248104 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:19.247879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:19.248104 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:19.248021 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:19.248205 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:19.248096 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:19.319165 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:19.319125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" event={"ID":"a22a6cc7-0a24-48ad-9943-70d15df02493","Type":"ContainerStarted","Data":"c8fa2585b547d1b976c491dc8270eae4921891c3fa3d9a0d7f0bef315a7627ac"} Jul 17 19:56:19.320692 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:19.320658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerStarted","Data":"ba2b12ee85c7b50468faf0428c8b6472e93b1f6c01f1697e687e8b6be99b3b2c"} Jul 17 19:56:20.247974 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:20.247935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:20.248164 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:20.248064 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:21.248170 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:21.248129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:21.248602 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:21.248129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:21.248602 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:21.248275 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:21.248602 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:21.248320 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:22.247242 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.247162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:22.247428 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.247329 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:22.328987 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.328946 2578 generic.go:356] "Generic (PLEG): container finished" podID="35e0fc6a14943475efe90a14e95086ef" containerID="ba2b12ee85c7b50468faf0428c8b6472e93b1f6c01f1697e687e8b6be99b3b2c" exitCode=0 Jul 17 19:56:22.329396 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.329053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerDied","Data":"ba2b12ee85c7b50468faf0428c8b6472e93b1f6c01f1697e687e8b6be99b3b2c"} Jul 17 19:56:22.331280 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.331252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jftrg" event={"ID":"67eb9374-9fb7-4b2f-8b35-eee4d7649140","Type":"ContainerStarted","Data":"f0e6570384ffb088dc9d501909054155634a939dea1741111a1fb92a8e852334"} Jul 17 19:56:22.367289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.367221 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jftrg" podStartSLOduration=7.105727517 podStartE2EDuration="17.367198533s" totalImagesPullingTime="10.261471016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:22.366787438 +0000 UTC m=+17.622018857" watchObservedRunningTime="2026-07-17 19:56:22.367198533 +0000 UTC m=+17.622429952" Jul 17 19:56:22.955260 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.955226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:22.955445 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.955277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:22.955445 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:22.955318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:22.955445 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955391 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:22.955445 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955406 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955449 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955468 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:56:38.955447369 +0000 UTC m=+34.210678764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955468 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955489 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955501 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:38.955484879 +0000 UTC m=+34.210716275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:22.955668 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:22.955524 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:38.955513582 +0000 UTC m=+34.210744991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:23.247467 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:23.247370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:23.247631 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:23.247524 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:23.247631 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:23.247587 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:23.247753 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:23.247706 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:24.247591 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:24.247555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:24.248023 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:24.247697 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:25.248744 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:25.248484 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:25.249197 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:25.248563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:25.249197 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:25.248839 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:25.249197 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:25.248877 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:26.247917 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:26.247886 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:26.248072 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:26.247986 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:26.911022 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:26.909346 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 19:56:27.189068 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.188983 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T19:56:26.909472077Z","UUID":"7fc7110e-9f22-4b12-a458-e6b42b091024","Handler":null,"Name":"","Endpoint":""} Jul 17 19:56:27.190522 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.190504 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 19:56:27.190667 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.190544 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 19:56:27.247878 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.247849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:27.248016 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:27.247959 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:27.248073 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.248018 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:27.248151 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:27.248132 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:27.341948 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.341924 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="faadbf09eaf4969b56d1698b2ea997eccf04c6fbbe5e8caaa015b0f9bea8ddf0" exitCode=0 Jul 17 19:56:27.342046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.341997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"faadbf09eaf4969b56d1698b2ea997eccf04c6fbbe5e8caaa015b0f9bea8ddf0"} Jul 17 19:56:27.343693 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.343639 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-87pgj" event={"ID":"002b2b02-4cf2-49f7-8ab6-cac22026c906","Type":"ContainerStarted","Data":"63facf0df17f20e246aaefdbe8b041ab61132f374ad30c9d51508a93f84c9dea"} Jul 17 19:56:27.345153 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.345117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cq8td" event={"ID":"a0f3d694-caca-4f28-b36a-0d7e36afb234","Type":"ContainerStarted","Data":"36d7e3f3b7140ae17f908cf6097f1103dddae68dc25b8712f91089c260e07a68"} Jul 17 19:56:27.346971 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.346945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" event={"ID":"a22a6cc7-0a24-48ad-9943-70d15df02493","Type":"ContainerStarted","Data":"9f06edf1b0eac514935dd21e56b87e6bbbfbd7fd90596adddd8537d16fff15dd"} Jul 17 19:56:27.349530 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.349438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"9fbc84a6d00e297d1bc496367c7684926c2151312fad36e2014edd0a039d0be3"} Jul 17 19:56:27.349530 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.349516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"baa015cec75e0e0e3916f927c2a7cd7ced39a187ae7fe26334840d40357e8efc"} Jul 17 19:56:27.349647 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.349565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"9bf2765312626efad6729afb59b3b95476b5a0204b8d4cf3cd1e38780ba0284a"} Jul 17 19:56:27.349647 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.349631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"915d685aa1389ff704988babdce54ee537ab6917c47552d8f3e6190e36449198"} Jul 17 19:56:27.351212 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.351191 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerStarted","Data":"a54f11674b00692deb9e341efbe666842a9cc07ef4187e986fc851b29d5c03f3"} Jul 17 19:56:27.352581 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.352557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8djmx" event={"ID":"3242d8ab-8ab7-4544-99e3-092ac4b7eef6","Type":"ContainerStarted","Data":"c8c3ae5a520d913c9f2dd2ce12d3f580cf22430d27011d0b608522604a783ad6"} Jul 17 19:56:27.353959 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.353937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfxzd" event={"ID":"8fbb3889-3c38-4343-a98f-0b5326c1a9f4","Type":"ContainerStarted","Data":"0616765e63484cd6efa6608c2588d911c74e6ea84d93290bf6796632558a1973"} Jul 17 19:56:27.398497 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.398449 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-87pgj" podStartSLOduration=3.195205671 podStartE2EDuration="22.398433418s" totalImagesPullingTime="19.203227747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:27.398015755 +0000 UTC m=+22.653247171" watchObservedRunningTime="2026-07-17 19:56:27.398433418 +0000 UTC m=+22.653664835" Jul 17 19:56:27.423079 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.423037 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" podStartSLOduration=21.423023544 podStartE2EDuration="21.423023544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:27.422930286 +0000 UTC m=+22.678161717" watchObservedRunningTime="2026-07-17 19:56:27.423023544 +0000 UTC m=+22.678254958" Jul 17 19:56:27.440016 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.439977 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cq8td" podStartSLOduration=12.155869746 podStartE2EDuration="22.43996296s" totalImagesPullingTime="10.284093214s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:27.439908152 +0000 UTC m=+22.695139568" watchObservedRunningTime="2026-07-17 19:56:27.43996296 +0000 UTC m=+22.695194376" Jul 17 19:56:27.491932 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:27.491841 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8djmx" podStartSLOduration=3.479451497 podStartE2EDuration="22.491827946s" totalImagesPullingTime="19.012376449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:27.466325049 +0000 UTC m=+22.721556466" watchObservedRunningTime="2026-07-17 19:56:27.491827946 +0000 UTC m=+22.747059363" Jul 17 19:56:28.248157 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.248134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:28.248514 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:28.248259 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:28.359997 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.359960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"6347cccd399d2bb675fe4b8acea94a14c8f70e82ff8dd2993dd3fd7a4ddd3ef6"} Jul 17 19:56:28.360177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.360058 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"7d6547ca0a17f31de57f7696677957d4c32f9bb04a33eace825aee5f953d529c"} Jul 17 19:56:28.361799 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.361633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-684c9" event={"ID":"ca287ec2-874f-42af-8a4b-d42a1a1dce8c","Type":"ContainerStarted","Data":"484154e2803376b65f50f9847528e538245fc76638fc2b12dfdb5fec61bfed8d"} Jul 17 19:56:28.403982 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.403922 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zfxzd" podStartSLOduration=13.110920868000001 podStartE2EDuration="23.40390114s" totalImagesPullingTime="10.292980272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:27.492162899 +0000 UTC m=+22.747394316" watchObservedRunningTime="2026-07-17 19:56:28.40390114 +0000 UTC m=+23.659132560" Jul 17 19:56:28.404235 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:28.404207 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-684c9" podStartSLOduration=4.173154861 podStartE2EDuration="23.404200331s" totalImagesPullingTime="19.23104547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:28.404046289 +0000 UTC m=+23.659277696" watchObservedRunningTime="2026-07-17 19:56:28.404200331 +0000 UTC m=+23.659431749" Jul 17 19:56:29.095053 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.095017 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:29.095736 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.095715 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:29.248160 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.248128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:29.248730 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.248128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:29.248730 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:29.248249 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:29.248730 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:29.248349 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:29.371456 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.371330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" event={"ID":"a22a6cc7-0a24-48ad-9943-70d15df02493","Type":"ContainerStarted","Data":"b2859d7d9ae144b1ef7d2d7be70edd8721e0c5624263e627a3bd646faec29b1e"} Jul 17 19:56:29.371778 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.371713 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:29.372061 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.372042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zfxzd" Jul 17 19:56:29.407357 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:29.407306 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrxzq" podStartSLOduration=4.825863284 podStartE2EDuration="24.407291345s" totalImagesPullingTime="19.581428061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:29.406488423 +0000 UTC m=+24.661719839" watchObservedRunningTime="2026-07-17 19:56:29.407291345 +0000 UTC m=+24.662522761" Jul 17 19:56:30.248140 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:30.248100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:30.248320 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:30.248242 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:30.376668 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:30.376446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"efe8963ba62bb0fcea26e807f98a4ebec703d34a86a4574b34499c1f9b3baf15"} Jul 17 19:56:31.247526 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:31.247468 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:31.247726 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:31.247472 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:31.247726 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:31.247635 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:31.247844 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:31.247745 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:32.247307 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:32.247275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:32.247701 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:32.247384 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:33.248021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.247830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:33.248857 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.247898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:33.248857 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:33.248121 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:33.248857 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:33.248152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:33.385338 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.385299 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="c0e4c7f0febd4c586eac67bc6094aedc2b12167cf1f651c6afca4091eb631d1e" exitCode=0 Jul 17 19:56:33.385565 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.385381 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"c0e4c7f0febd4c586eac67bc6094aedc2b12167cf1f651c6afca4091eb631d1e"} Jul 17 19:56:33.389069 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.389048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" event={"ID":"47f1e417-214a-49d4-aca6-7b0bb1fc0627","Type":"ContainerStarted","Data":"50ca64f1dab49a25b5b26935cef59eb3ebb64b2559b4da83661a3c09dae0d6b5"} Jul 17 19:56:33.389311 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.389294 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:33.389507 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.389351 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:33.389507 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.389400 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:33.404426 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.404399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:33.404558 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.404497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:56:33.436841 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:33.436791 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" podStartSLOduration=9.793029532 podStartE2EDuration="27.436773585s" totalImagesPullingTime="17.643744053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:33.436695304 +0000 UTC m=+28.691926721" watchObservedRunningTime="2026-07-17 19:56:33.436773585 +0000 UTC m=+28.692005049" Jul 17 19:56:34.247535 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.247505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:34.247720 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:34.247665 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:34.394244 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.394208 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="1e83fc5fc0cd1f470443e49c71eda4354021658b8cf096d89b98e7718fbba8c6" exitCode=0 Jul 17 19:56:34.394768 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.394308 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"1e83fc5fc0cd1f470443e49c71eda4354021658b8cf096d89b98e7718fbba8c6"} Jul 17 19:56:34.795462 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.795432 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqfrh"] Jul 17 19:56:34.795626 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.795541 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:34.795673 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:34.795659 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:34.797429 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.797396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-srdd9"] Jul 17 19:56:34.797629 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.797514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:34.797705 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:34.797634 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:34.798119 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.798089 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8gscn"] Jul 17 19:56:34.798224 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:34.798186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:34.798298 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:34.798276 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:36.247506 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:36.247318 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:36.248023 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:36.247324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:36.248023 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:36.247593 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:36.248023 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:36.247364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:36.248023 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:36.247695 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:36.248023 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:36.247786 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:36.400742 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:36.400715 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="2ebea6b7df3ed916e4792b15f70c47f79d2e49525111809fa83dcbe7104ae0ae" exitCode=0 Jul 17 19:56:36.400742 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:36.400747 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"2ebea6b7df3ed916e4792b15f70c47f79d2e49525111809fa83dcbe7104ae0ae"} Jul 17 19:56:38.247666 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.247626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:38.247666 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.247658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:38.247666 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.247674 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:38.248378 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.247793 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqfrh" podUID="908d7010-dec8-4574-9ce4-29a5d5327bec" Jul 17 19:56:38.248378 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.248070 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-srdd9" podUID="bdddb632-f19c-48ad-83a9-7ca487182630" Jul 17 19:56:38.248378 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.248116 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gscn" podUID="1d6ecb91-80ae-4221-9797-50fa1be7eda1" Jul 17 19:56:38.965234 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.965198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.965244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:38.965273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965348 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965357 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965366 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965409 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret podName:908d7010-dec8-4574-9ce4-29a5d5327bec nodeName:}" failed. No retries permitted until 2026-07-17 19:57:10.965393384 +0000 UTC m=+66.220624782 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret") pod "global-pull-secret-syncer-rqfrh" (UID: "908d7010-dec8-4574-9ce4-29a5d5327bec") : object "kube-system"/"original-pull-secret" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965423 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs podName:1d6ecb91-80ae-4221-9797-50fa1be7eda1 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:10.965416609 +0000 UTC m=+66.220648003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs") pod "network-metrics-daemon-8gscn" (UID: "1d6ecb91-80ae-4221-9797-50fa1be7eda1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965424 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 19:56:38.965435 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965438 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6npxv for pod openshift-network-diagnostics/network-check-target-srdd9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:38.965789 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:38.965477 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv podName:bdddb632-f19c-48ad-83a9-7ca487182630 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:10.965461989 +0000 UTC m=+66.220693393 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6npxv" (UniqueName: "kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv") pod "network-check-target-srdd9" (UID: "bdddb632-f19c-48ad-83a9-7ca487182630") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 19:56:39.316422 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.316397 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeReady" Jul 17 19:56:39.316770 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.316518 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 19:56:39.356775 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.356745 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:56:39.360465 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.360436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.362812 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.362791 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wqfls\"" Jul 17 19:56:39.362940 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.362787 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 19:56:39.363510 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.363488 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 19:56:39.363868 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.363848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 19:56:39.368635 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.368487 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 19:56:39.370574 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.370553 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g"] Jul 17 19:56:39.374282 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.374261 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-2v946"] Jul 17 19:56:39.374418 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.374395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" Jul 17 19:56:39.376739 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.376719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.377218 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.377200 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.377298 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.377207 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zt7tn\"" Jul 17 19:56:39.377463 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.377444 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-7grmk"] Jul 17 19:56:39.377577 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.377563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.380306 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380286 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 17 19:56:39.380390 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380288 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4fvcm\"" Jul 17 19:56:39.380390 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380294 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 17 19:56:39.380557 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380543 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9"] Jul 17 19:56:39.380689 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380668 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.381058 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.380884 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.385233 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.385208 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.385347 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.385296 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.385904 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.385874 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.386542 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.386198 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9h79c\"" Jul 17 19:56:39.386542 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.386263 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 17 19:56:39.390403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.388092 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j"] Jul 17 19:56:39.390403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.389421 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.390403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.389540 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 17 19:56:39.395057 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.394302 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.395469 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.395444 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-765ff78b4-jfc5r"] Jul 17 19:56:39.398287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.396514 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 17 19:56:39.398287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.396781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.398287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.396885 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.398287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.397064 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-98hb8\"" Jul 17 19:56:39.398287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.397426 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 17 19:56:39.398699 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.398533 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 17 19:56:39.398819 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.398804 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48"] Jul 17 19:56:39.399454 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.399226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.403386 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.402738 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k"] Jul 17 19:56:39.403386 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.402936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.404690 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.404668 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 17 19:56:39.404872 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.404853 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.404973 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.404894 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 17 19:56:39.405095 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405078 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 17 19:56:39.405159 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405139 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.405286 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t24ss\"" Jul 17 19:56:39.405370 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405325 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 17 19:56:39.405533 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405514 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 17 19:56:39.405645 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405630 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.405882 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.405865 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 17 19:56:39.406040 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406027 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 17 19:56:39.406087 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406060 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qmmcs\"" Jul 17 19:56:39.406269 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.406441 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.406665 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406601 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:56:39.406767 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406682 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-7grmk"] Jul 17 19:56:39.406767 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9"] Jul 17 19:56:39.406767 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g"] Jul 17 19:56:39.406767 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48"] Jul 17 19:56:39.406767 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k"] Jul 17 19:56:39.407003 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406781 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-765ff78b4-jfc5r"] Jul 17 19:56:39.407003 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406808 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j"] Jul 17 19:56:39.407003 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406829 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8pkg4"] Jul 17 19:56:39.407003 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.406780 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.409580 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.409560 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hqsst\"" Jul 17 19:56:39.410967 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.410947 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.411042 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.410965 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4"] Jul 17 19:56:39.411077 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411042 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.411122 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411096 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 17 19:56:39.411334 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4rszd\"" Jul 17 19:56:39.411385 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411339 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.411426 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411317 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 17 19:56:39.411768 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.411750 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.412534 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.412518 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 17 19:56:39.414346 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.414323 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zsckk"] Jul 17 19:56:39.414466 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.414451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.415840 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.415820 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkppg\"" Jul 17 19:56:39.415947 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.415820 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 19:56:39.416044 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.416030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 19:56:39.417398 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.417381 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn"] Jul 17 19:56:39.417510 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.417495 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.422457 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.421626 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 17 19:56:39.422457 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.421685 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 17 19:56:39.422457 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.421975 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5rcld\"" Jul 17 19:56:39.422457 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.422071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" Jul 17 19:56:39.424778 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.422581 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c78df\"" Jul 17 19:56:39.424778 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.423163 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.424778 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.423707 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 19:56:39.424778 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.424280 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.427263 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.426678 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 19:56:39.428225 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.428205 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 19:56:39.428836 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.428813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ff89v\"" Jul 17 19:56:39.429401 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.429153 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-2v946"] Jul 17 19:56:39.430582 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.430561 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn"] Jul 17 19:56:39.432348 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.431898 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4"] Jul 17 19:56:39.433012 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.432990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zsckk"] Jul 17 19:56:39.434126 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.434097 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8pkg4"] Jul 17 19:56:39.468834 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-stats-auth\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.468981 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.468981 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eba72d2c-0061-47b3-9339-2afcfebc950b-serving-cert\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.468981 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c210c3d-d346-4651-904a-779637bc420b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.468981 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlvbv\" (UniqueName: \"kubernetes.io/projected/eba72d2c-0061-47b3-9339-2afcfebc950b-kube-api-access-rlvbv\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.469175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.468997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.469175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slr79\" (UniqueName: \"kubernetes.io/projected/2c210c3d-d346-4651-904a-779637bc420b-kube-api-access-slr79\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.469175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-service-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-tmp\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20a98849-040d-4d02-8485-70ed0281cf4f-serving-cert\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c210c3d-d346-4651-904a-779637bc420b-config\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqk4w\" (UniqueName: \"kubernetes.io/projected/3ef3b898-af9f-41a1-8139-39e429a12416-kube-api-access-fqk4w\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469269 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt4hb\" (UniqueName: \"kubernetes.io/projected/2975d364-f194-463a-8924-a1881774eb30-kube-api-access-mt4hb\") pod \"volume-data-source-validator-78cf4c6f6-6w47g\" (UID: \"2975d364-f194-463a-8924-a1881774eb30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-snapshots\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.469410 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-config\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9r6\" (UniqueName: \"kubernetes.io/projected/20a98849-040d-4d02-8485-70ed0281cf4f-kube-api-access-wb9r6\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc8dn\" (UniqueName: \"kubernetes.io/projected/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-kube-api-access-zc8dn\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.469776 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-default-certificate\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.470105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.470105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-trusted-ca\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.470105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp5jb\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.470105 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.469885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.570367 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9r6\" (UniqueName: \"kubernetes.io/projected/20a98849-040d-4d02-8485-70ed0281cf4f-kube-api-access-wb9r6\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.570367 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.570367 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.570367 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570363 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4978d\" (UniqueName: \"kubernetes.io/projected/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-kube-api-access-4978d\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1be32d05-bf30-4d24-9ce0-483471ef371c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc8dn\" (UniqueName: \"kubernetes.io/projected/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-kube-api-access-zc8dn\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q566z\" (UniqueName: \"kubernetes.io/projected/1be32d05-bf30-4d24-9ce0-483471ef371c-kube-api-access-q566z\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-default-certificate\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.570622 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.070585795 +0000 UTC m=+35.325817200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.570696 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:39.570741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-trusted-ca\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp5jb\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.570796 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.570837 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.07081946 +0000 UTC m=+35.326050854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34abb3e7-a612-4773-9220-b47320b4ce85-cert\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-stats-auth\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-tmp-dir\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.570979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6pt5\" (UniqueName: \"kubernetes.io/projected/34abb3e7-a612-4773-9220-b47320b4ce85-kube-api-access-c6pt5\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqj2v\" (UniqueName: \"kubernetes.io/projected/c014a045-d831-4d06-b875-f4460634be03-kube-api-access-hqj2v\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eba72d2c-0061-47b3-9339-2afcfebc950b-serving-cert\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c210c3d-d346-4651-904a-779637bc420b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlvbv\" (UniqueName: \"kubernetes.io/projected/eba72d2c-0061-47b3-9339-2afcfebc950b-kube-api-access-rlvbv\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571164 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.571333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slr79\" (UniqueName: \"kubernetes.io/projected/2c210c3d-d346-4651-904a-779637bc420b-kube-api-access-slr79\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-service-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-config-volume\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-tmp\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96p9v\" (UniqueName: \"kubernetes.io/projected/26897308-b349-4bde-8256-8ef913646965-kube-api-access-96p9v\") pod \"network-check-source-7bc5b6bf85-ljksn\" (UID: \"26897308-b349-4bde-8256-8ef913646965\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.571667 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.071644817 +0000 UTC m=+35.326876224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20a98849-040d-4d02-8485-70ed0281cf4f-serving-cert\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.572110 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.571737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-trusted-ca\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.572730 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572707 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.572822 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572768 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.572822 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c210c3d-d346-4651-904a-779637bc420b-config\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.572921 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-tmp\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.572921 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1be32d05-bf30-4d24-9ce0-483471ef371c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.573010 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqk4w\" (UniqueName: \"kubernetes.io/projected/3ef3b898-af9f-41a1-8139-39e429a12416-kube-api-access-fqk4w\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.573010 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt4hb\" (UniqueName: \"kubernetes.io/projected/2975d364-f194-463a-8924-a1881774eb30-kube-api-access-mt4hb\") pod \"volume-data-source-validator-78cf4c6f6-6w47g\" (UID: \"2975d364-f194-463a-8924-a1881774eb30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" Jul 17 19:56:39.573010 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.572987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c014a045-d831-4d06-b875-f4460634be03-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.573156 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-snapshots\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.573210 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a98849-040d-4d02-8485-70ed0281cf4f-service-ca-bundle\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.573210 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.573356 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-config\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.573356 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c210c3d-d346-4651-904a-779637bc420b-config\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.573356 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/26b6965a-9faa-468e-9d98-0c63b8126a86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.573503 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.573370 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:39.573503 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.573383 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:39.573503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.573503 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.573456 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.073438032 +0000 UTC m=+35.328669428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:39.573743 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573641 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20a98849-040d-4d02-8485-70ed0281cf4f-snapshots\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.574014 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.573968 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eba72d2c-0061-47b3-9339-2afcfebc950b-config\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.574356 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.574331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.575661 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.575640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20a98849-040d-4d02-8485-70ed0281cf4f-serving-cert\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.575777 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.575739 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.576016 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.575986 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eba72d2c-0061-47b3-9339-2afcfebc950b-serving-cert\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.576135 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.576116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.576176 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.576133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-stats-auth\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.576211 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.576182 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c210c3d-d346-4651-904a-779637bc420b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.576357 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.576342 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-default-certificate\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.583665 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.583644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9r6\" (UniqueName: \"kubernetes.io/projected/20a98849-040d-4d02-8485-70ed0281cf4f-kube-api-access-wb9r6\") pod \"insights-operator-555d9d7885-2v946\" (UID: \"20a98849-040d-4d02-8485-70ed0281cf4f\") " pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.586954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.586922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt4hb\" (UniqueName: \"kubernetes.io/projected/2975d364-f194-463a-8924-a1881774eb30-kube-api-access-mt4hb\") pod \"volume-data-source-validator-78cf4c6f6-6w47g\" (UID: \"2975d364-f194-463a-8924-a1881774eb30\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" Jul 17 19:56:39.587117 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.587096 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slr79\" (UniqueName: \"kubernetes.io/projected/2c210c3d-d346-4651-904a-779637bc420b-kube-api-access-slr79\") pod \"service-ca-operator-5bcbc94c78-6mg2j\" (UID: \"2c210c3d-d346-4651-904a-779637bc420b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.587368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.587345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlvbv\" (UniqueName: \"kubernetes.io/projected/eba72d2c-0061-47b3-9339-2afcfebc950b-kube-api-access-rlvbv\") pod \"console-operator-565b7b569d-7grmk\" (UID: \"eba72d2c-0061-47b3-9339-2afcfebc950b\") " pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.587875 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.587856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp5jb\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.588503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.588481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc8dn\" (UniqueName: \"kubernetes.io/projected/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-kube-api-access-zc8dn\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:39.588741 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.588718 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:39.589480 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.589449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqk4w\" (UniqueName: \"kubernetes.io/projected/3ef3b898-af9f-41a1-8139-39e429a12416-kube-api-access-fqk4w\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:39.674338 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674308 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-config-volume\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96p9v\" (UniqueName: \"kubernetes.io/projected/26897308-b349-4bde-8256-8ef913646965-kube-api-access-96p9v\") pod \"network-check-source-7bc5b6bf85-ljksn\" (UID: \"26897308-b349-4bde-8256-8ef913646965\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1be32d05-bf30-4d24-9ce0-483471ef371c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c014a045-d831-4d06-b875-f4460634be03-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/26b6965a-9faa-468e-9d98-0c63b8126a86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.674503 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.674471 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4978d\" (UniqueName: \"kubernetes.io/projected/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-kube-api-access-4978d\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.674546 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.174527839 +0000 UTC m=+35.429759242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1be32d05-bf30-4d24-9ce0-483471ef371c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q566z\" (UniqueName: \"kubernetes.io/projected/1be32d05-bf30-4d24-9ce0-483471ef371c-kube-api-access-q566z\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34abb3e7-a612-4773-9220-b47320b4ce85-cert\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-tmp-dir\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.674863 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6pt5\" (UniqueName: \"kubernetes.io/projected/34abb3e7-a612-4773-9220-b47320b4ce85-kube-api-access-c6pt5\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqj2v\" (UniqueName: \"kubernetes.io/projected/c014a045-d831-4d06-b875-f4460634be03-kube-api-access-hqj2v\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.674985 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-config-volume\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.675040 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1be32d05-bf30-4d24-9ce0-483471ef371c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.674853 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.675112 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.175097957 +0000 UTC m=+35.430329354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.675115 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:39.675265 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:39.675162 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:40.175148246 +0000 UTC m=+35.430379642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:39.675589 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.675408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/26b6965a-9faa-468e-9d98-0c63b8126a86-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:39.675589 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.675476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-tmp-dir\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.676038 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.676011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c014a045-d831-4d06-b875-f4460634be03-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.677918 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.677896 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1be32d05-bf30-4d24-9ce0-483471ef371c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.678065 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.678049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34abb3e7-a612-4773-9220-b47320b4ce85-cert\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.687322 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.686531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q566z\" (UniqueName: \"kubernetes.io/projected/1be32d05-bf30-4d24-9ce0-483471ef371c-kube-api-access-q566z\") pod \"kube-storage-version-migrator-operator-7658d9b749-g6c7k\" (UID: \"1be32d05-bf30-4d24-9ce0-483471ef371c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.687441 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.687405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqj2v\" (UniqueName: \"kubernetes.io/projected/c014a045-d831-4d06-b875-f4460634be03-kube-api-access-hqj2v\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:39.687520 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.687506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" Jul 17 19:56:39.688799 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.688413 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6pt5\" (UniqueName: \"kubernetes.io/projected/34abb3e7-a612-4773-9220-b47320b4ce85-kube-api-access-c6pt5\") pod \"ingress-canary-zsckk\" (UID: \"34abb3e7-a612-4773-9220-b47320b4ce85\") " pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.691011 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.690986 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4978d\" (UniqueName: \"kubernetes.io/projected/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-kube-api-access-4978d\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:39.691309 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.691288 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96p9v\" (UniqueName: \"kubernetes.io/projected/26897308-b349-4bde-8256-8ef913646965-kube-api-access-96p9v\") pod \"network-check-source-7bc5b6bf85-ljksn\" (UID: \"26897308-b349-4bde-8256-8ef913646965\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" Jul 17 19:56:39.704455 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.704434 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-2v946" Jul 17 19:56:39.713169 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.713147 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:39.731127 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.731067 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" Jul 17 19:56:39.754996 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.754948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" Jul 17 19:56:39.776499 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.776075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zsckk" Jul 17 19:56:39.785394 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.785308 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" Jul 17 19:56:39.943153 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:39.943074 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2975d364_f194_463a_8924_a1881774eb30.slice/crio-dd1217f93ca2560f7110ad70f42ee4adb99a5cd9b63bee515bd04d9ceb6739dc WatchSource:0}: Error finding container dd1217f93ca2560f7110ad70f42ee4adb99a5cd9b63bee515bd04d9ceb6739dc: Status 404 returned error can't find the container with id dd1217f93ca2560f7110ad70f42ee4adb99a5cd9b63bee515bd04d9ceb6739dc Jul 17 19:56:39.943153 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.943130 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g"] Jul 17 19:56:39.956317 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.956240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-2v946"] Jul 17 19:56:39.968991 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.968918 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-7grmk"] Jul 17 19:56:39.974967 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:39.974934 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba72d2c_0061_47b3_9339_2afcfebc950b.slice/crio-083a4c58c867f5a7ff3d421865342248e9af33b5f5d2e1205728c52393712e73 WatchSource:0}: Error finding container 083a4c58c867f5a7ff3d421865342248e9af33b5f5d2e1205728c52393712e73: Status 404 returned error can't find the container with id 083a4c58c867f5a7ff3d421865342248e9af33b5f5d2e1205728c52393712e73 Jul 17 19:56:39.993508 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:39.993468 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j"] Jul 17 19:56:39.996483 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:39.996459 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c210c3d_d346_4651_904a_779637bc420b.slice/crio-1f613af10a11b1da5ce9cbf9c5f698680ef54a9e6270e5b622189a65c4767946 WatchSource:0}: Error finding container 1f613af10a11b1da5ce9cbf9c5f698680ef54a9e6270e5b622189a65c4767946: Status 404 returned error can't find the container with id 1f613af10a11b1da5ce9cbf9c5f698680ef54a9e6270e5b622189a65c4767946 Jul 17 19:56:40.003748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.003681 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k"] Jul 17 19:56:40.006543 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:40.006514 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be32d05_bf30_4d24_9ce0_483471ef371c.slice/crio-b97dc7d6869b67a267c0b5b1ed6d47df29151dda991ec7b8e2e5b2dad97b5596 WatchSource:0}: Error finding container b97dc7d6869b67a267c0b5b1ed6d47df29151dda991ec7b8e2e5b2dad97b5596: Status 404 returned error can't find the container with id b97dc7d6869b67a267c0b5b1ed6d47df29151dda991ec7b8e2e5b2dad97b5596 Jul 17 19:56:40.019084 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.019059 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zsckk"] Jul 17 19:56:40.023746 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:40.023722 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34abb3e7_a612_4773_9220_b47320b4ce85.slice/crio-5aaf5d7168f08b52838abbffb37353fe532cdfcead21d442764067486653ed28 WatchSource:0}: Error finding container 5aaf5d7168f08b52838abbffb37353fe532cdfcead21d442764067486653ed28: Status 404 returned error can't find the container with id 5aaf5d7168f08b52838abbffb37353fe532cdfcead21d442764067486653ed28 Jul 17 19:56:40.029270 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.029207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn"] Jul 17 19:56:40.031741 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:56:40.031715 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26897308_b349_4bde_8256_8ef913646965.slice/crio-d734f32b74388dba05132ce8a08213f2ead1599ef8d0959d641cde2be89864ce WatchSource:0}: Error finding container d734f32b74388dba05132ce8a08213f2ead1599ef8d0959d641cde2be89864ce: Status 404 returned error can't find the container with id d734f32b74388dba05132ce8a08213f2ead1599ef8d0959d641cde2be89864ce Jul 17 19:56:40.084606 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.084520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:40.084606 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.084562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:40.084606 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.084590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:40.084886 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.084744 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.084723259 +0000 UTC m=+36.339954664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:40.084886 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.084805 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:40.084886 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.084858 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:40.085089 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.085066 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.085049674 +0000 UTC m=+36.340281074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:40.085204 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.085114 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.085103637 +0000 UTC m=+36.340335072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:40.085204 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.085163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:40.085297 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.085259 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:40.085297 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.085271 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:40.085368 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.085314 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.085306423 +0000 UTC m=+36.340537819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:40.186182 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.186155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:40.186368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.186222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:40.186368 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186318 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:40.186368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.186340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:40.186524 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186378 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.18636404 +0000 UTC m=+36.441595434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:40.186524 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186382 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:40.186524 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186428 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:40.186524 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186439 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.186423159 +0000 UTC m=+36.441654554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:40.186524 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:40.186471 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:41.186457747 +0000 UTC m=+36.441689145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:40.248296 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.248263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:56:40.248473 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.248268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:56:40.248473 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.248274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:56:40.250909 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.250875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 19:56:40.251046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.250939 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdp2d\"" Jul 17 19:56:40.251046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.251001 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 19:56:40.251046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.251015 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clqd4\"" Jul 17 19:56:40.413310 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.413274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zsckk" event={"ID":"34abb3e7-a612-4773-9220-b47320b4ce85","Type":"ContainerStarted","Data":"5aaf5d7168f08b52838abbffb37353fe532cdfcead21d442764067486653ed28"} Jul 17 19:56:40.414491 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.414466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" event={"ID":"2975d364-f194-463a-8924-a1881774eb30","Type":"ContainerStarted","Data":"dd1217f93ca2560f7110ad70f42ee4adb99a5cd9b63bee515bd04d9ceb6739dc"} Jul 17 19:56:40.415648 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.415605 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" event={"ID":"1be32d05-bf30-4d24-9ce0-483471ef371c","Type":"ContainerStarted","Data":"b97dc7d6869b67a267c0b5b1ed6d47df29151dda991ec7b8e2e5b2dad97b5596"} Jul 17 19:56:40.416840 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.416817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" event={"ID":"26897308-b349-4bde-8256-8ef913646965","Type":"ContainerStarted","Data":"d734f32b74388dba05132ce8a08213f2ead1599ef8d0959d641cde2be89864ce"} Jul 17 19:56:40.417958 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.417935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" event={"ID":"2c210c3d-d346-4651-904a-779637bc420b","Type":"ContainerStarted","Data":"1f613af10a11b1da5ce9cbf9c5f698680ef54a9e6270e5b622189a65c4767946"} Jul 17 19:56:40.419037 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.419017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" event={"ID":"eba72d2c-0061-47b3-9339-2afcfebc950b","Type":"ContainerStarted","Data":"083a4c58c867f5a7ff3d421865342248e9af33b5f5d2e1205728c52393712e73"} Jul 17 19:56:40.420118 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:40.420100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-2v946" event={"ID":"20a98849-040d-4d02-8485-70ed0281cf4f","Type":"ContainerStarted","Data":"2580902e00106a847f90c597075ab3c6bb6033f31a0cda9bc6182c3f8c723f31"} Jul 17 19:56:41.095591 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.095551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:41.095785 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.095650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:41.095785 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.095676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:41.095785 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.095721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:41.095785 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095734 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:41.095785 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095756 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095802 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.095794584 +0000 UTC m=+38.351025984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095850 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095852 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.095833807 +0000 UTC m=+38.351065208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095877 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.095867779 +0000 UTC m=+38.351099174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:41.096040 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.095912 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.095886238 +0000 UTC m=+38.351117647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:41.197196 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.197159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:41.197382 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.197245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:41.197382 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197322 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:41.197382 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:41.197360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:41.197529 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197390 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.197372207 +0000 UTC m=+38.452603602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:41.197529 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197400 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:41.197529 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197420 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:41.197529 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197467 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.197451213 +0000 UTC m=+38.452682618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:41.197529 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:41.197492 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:43.197482273 +0000 UTC m=+38.452713670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:42.439062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:42.438821 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="df16eee340dc874d444853b60761dfa7214a2dd1697086aba84c1b3f2b6f0543" exitCode=0 Jul 17 19:56:42.439062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:42.438916 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"df16eee340dc874d444853b60761dfa7214a2dd1697086aba84c1b3f2b6f0543"} Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.126745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.126824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.126853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.126882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127118 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127179 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.127159908 +0000 UTC m=+42.382391305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127252 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127264 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127299 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.127288143 +0000 UTC m=+42.382519542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127363 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.12735252 +0000 UTC m=+42.382583920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127421 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:43.127783 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.127453 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.127442982 +0000 UTC m=+42.382674380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:43.227590 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.227551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:43.227750 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.227656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:43.227750 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227709 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:43.227880 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227781 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.227760408 +0000 UTC m=+42.482991823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:43.227880 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227790 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:43.227880 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227847 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.227829877 +0000 UTC m=+42.483061274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:43.228034 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227906 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:43.228034 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:43.227939 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:47.227927899 +0000 UTC m=+42.483159327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:43.228034 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.227712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:43.454404 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:43.454367 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerStarted","Data":"ef00db5083ba81b82791410e879c79e0191ea96eb99671f408e59f40e2f48b7f"} Jul 17 19:56:44.460541 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:44.460345 2578 generic.go:356] "Generic (PLEG): container finished" podID="a69bc2e5-8958-461f-a7e9-dca1c0c070c5" containerID="ef00db5083ba81b82791410e879c79e0191ea96eb99671f408e59f40e2f48b7f" exitCode=0 Jul 17 19:56:44.461026 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:44.460422 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerDied","Data":"ef00db5083ba81b82791410e879c79e0191ea96eb99671f408e59f40e2f48b7f"} Jul 17 19:56:44.462482 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:44.462458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" event={"ID":"2975d364-f194-463a-8924-a1881774eb30","Type":"ContainerStarted","Data":"7de532b70682001030bfa0b485946caf6ac028cad5aba7e134183192bff83957"} Jul 17 19:56:44.499623 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:44.499552 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6w47g" podStartSLOduration=31.97902037 podStartE2EDuration="35.499533115s" totalImagesPullingTime="3.520512745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:44.49847957 +0000 UTC m=+39.753711067" watchObservedRunningTime="2026-07-17 19:56:44.499533115 +0000 UTC m=+39.754764534" Jul 17 19:56:47.165518 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.165477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.165566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.165595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.165642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165666 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165684 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165733 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.165719789 +0000 UTC m=+50.420951189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165748 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.165741442 +0000 UTC m=+50.420972837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165793 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165822 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165860 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.165849214 +0000 UTC m=+50.421080612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:47.165989 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.165874 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.16586587 +0000 UTC m=+50.421097272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:47.266930 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.266892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:47.267121 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.266960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:47.267121 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:47.267020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:47.267121 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267050 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:47.267121 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267106 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:47.267121 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267115 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:47.267388 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267136 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.267117105 +0000 UTC m=+50.522348511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:47.267388 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267154 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.267145563 +0000 UTC m=+50.522376964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:47.267388 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:47.267175 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:56:55.26715907 +0000 UTC m=+50.522390466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:49.481922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.481888 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" event={"ID":"a69bc2e5-8958-461f-a7e9-dca1c0c070c5","Type":"ContainerStarted","Data":"1d6cc75d93f1a90c2066216484a927679c3fc0ba67ee0228e797072f9944b843"} Jul 17 19:56:49.483922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.483522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" event={"ID":"26897308-b349-4bde-8256-8ef913646965","Type":"ContainerStarted","Data":"4783d32cc9ecb63870046cc9764ee1006013bc6a8868f2203681089b95569a9c"} Jul 17 19:56:49.485083 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.485057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" event={"ID":"2c210c3d-d346-4651-904a-779637bc420b","Type":"ContainerStarted","Data":"52e408811501cc283f74200aa8dea4543bb444bc91e5b807f79b93450a349c77"} Jul 17 19:56:49.486806 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.486787 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/0.log" Jul 17 19:56:49.486888 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.486824 2578 generic.go:356] "Generic (PLEG): container finished" podID="eba72d2c-0061-47b3-9339-2afcfebc950b" containerID="fd8548ae511af16a4712974e87e218153e59c8e4b87a752dd509064df2522e7d" exitCode=255 Jul 17 19:56:49.486888 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.486858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" event={"ID":"eba72d2c-0061-47b3-9339-2afcfebc950b","Type":"ContainerDied","Data":"fd8548ae511af16a4712974e87e218153e59c8e4b87a752dd509064df2522e7d"} Jul 17 19:56:49.487076 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.487055 2578 scope.go:117] "RemoveContainer" containerID="fd8548ae511af16a4712974e87e218153e59c8e4b87a752dd509064df2522e7d" Jul 17 19:56:49.489846 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.489822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-2v946" event={"ID":"20a98849-040d-4d02-8485-70ed0281cf4f","Type":"ContainerStarted","Data":"45287d5ab5508ef4698c2cbd739a960f76c99d94637120bf114584263af8abac"} Jul 17 19:56:49.491402 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.491350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zsckk" event={"ID":"34abb3e7-a612-4773-9220-b47320b4ce85","Type":"ContainerStarted","Data":"48ffc66136d9c446d89f4fba75b46f5b773ead8d3bf6af1435cbf386ede2f1a5"} Jul 17 19:56:49.492923 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.492902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" event={"ID":"1be32d05-bf30-4d24-9ce0-483471ef371c","Type":"ContainerStarted","Data":"e2093f8d5f4819d0d307aab692a26d5e1cced4c9a7c10ffb93f6037cf1307033"} Jul 17 19:56:49.513772 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.513720 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dz8sz" podStartSLOduration=13.209390206 podStartE2EDuration="44.513700498s" totalImagesPullingTime="31.304310292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.513310792 +0000 UTC m=+44.768542209" watchObservedRunningTime="2026-07-17 19:56:49.513700498 +0000 UTC m=+44.768931918" Jul 17 19:56:49.542742 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.541218 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" podStartSLOduration=31.533913592 podStartE2EDuration="40.541198831s" totalImagesPullingTime="9.007285239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.535566758 +0000 UTC m=+44.790798179" watchObservedRunningTime="2026-07-17 19:56:49.541198831 +0000 UTC m=+44.796430249" Jul 17 19:56:49.564864 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.564809 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zsckk" podStartSLOduration=1.5842385829999999 podStartE2EDuration="10.564793069s" totalImagesPullingTime="8.980554486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.563496921 +0000 UTC m=+44.818728344" watchObservedRunningTime="2026-07-17 19:56:49.564793069 +0000 UTC m=+44.820024485" Jul 17 19:56:49.595109 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.595017 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-2v946" podStartSLOduration=31.554318301 podStartE2EDuration="40.594997045s" totalImagesPullingTime="9.040678744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.591330642 +0000 UTC m=+44.846562060" watchObservedRunningTime="2026-07-17 19:56:49.594997045 +0000 UTC m=+44.850228464" Jul 17 19:56:49.631257 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.631204 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ljksn" podStartSLOduration=31.647836147 podStartE2EDuration="40.631181634s" totalImagesPullingTime="8.983345487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.630602302 +0000 UTC m=+44.885833722" watchObservedRunningTime="2026-07-17 19:56:49.631181634 +0000 UTC m=+44.886413030" Jul 17 19:56:49.680881 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.680822 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" podStartSLOduration=31.683033492 podStartE2EDuration="40.680806897s" totalImagesPullingTime="8.997773405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:56:49.679388926 +0000 UTC m=+44.934620343" watchObservedRunningTime="2026-07-17 19:56:49.680806897 +0000 UTC m=+44.936038335" Jul 17 19:56:49.714185 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.714155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:49.714322 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:49.714261 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:50.498642 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.498585 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/1.log" Jul 17 19:56:50.499132 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.499112 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/0.log" Jul 17 19:56:50.499188 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.499156 2578 generic.go:356] "Generic (PLEG): container finished" podID="eba72d2c-0061-47b3-9339-2afcfebc950b" containerID="9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36" exitCode=255 Jul 17 19:56:50.499304 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.499278 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" event={"ID":"eba72d2c-0061-47b3-9339-2afcfebc950b","Type":"ContainerDied","Data":"9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36"} Jul 17 19:56:50.499363 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.499341 2578 scope.go:117] "RemoveContainer" containerID="fd8548ae511af16a4712974e87e218153e59c8e4b87a752dd509064df2522e7d" Jul 17 19:56:50.499681 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.499465 2578 scope.go:117] "RemoveContainer" containerID="9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36" Jul 17 19:56:50.499817 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:50.499693 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-7grmk_openshift-console-operator(eba72d2c-0061-47b3-9339-2afcfebc950b)\"" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podUID="eba72d2c-0061-47b3-9339-2afcfebc950b" Jul 17 19:56:50.731049 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.731020 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2wrtl"] Jul 17 19:56:50.764239 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.764155 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wrtl"] Jul 17 19:56:50.764395 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.764297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.767319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.767297 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 19:56:50.767484 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.767426 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 19:56:50.767557 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.767511 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nmtpw\"" Jul 17 19:56:50.802438 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.802410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ff80eb7-0b29-44a8-ab00-501321c6542e-crio-socket\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.802654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.802473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ff80eb7-0b29-44a8-ab00-501321c6542e-data-volume\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.802654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.802519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.802654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.802540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkdb\" (UniqueName: \"kubernetes.io/projected/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-api-access-ktkdb\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.802654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.802622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903542 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ff80eb7-0b29-44a8-ab00-501321c6542e-data-volume\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903542 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903794 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkdb\" (UniqueName: \"kubernetes.io/projected/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-api-access-ktkdb\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903794 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:50.903687 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 19:56:50.903794 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903794 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:50.903764 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:51.403742177 +0000 UTC m=+46.658973574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : configmap "kube-rbac-proxy" not found Jul 17 19:56:50.903973 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:50.903830 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 19:56:50.903973 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:50.903891 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:51.403874145 +0000 UTC m=+46.659105557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : secret "insights-runtime-extractor-tls" not found Jul 17 19:56:50.903973 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ff80eb7-0b29-44a8-ab00-501321c6542e-crio-socket\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.903973 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.903945 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ff80eb7-0b29-44a8-ab00-501321c6542e-data-volume\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.904108 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.904075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ff80eb7-0b29-44a8-ab00-501321c6542e-crio-socket\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:50.922419 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:50.922390 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkdb\" (UniqueName: \"kubernetes.io/projected/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-api-access-ktkdb\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:51.408629 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:51.408574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:51.408839 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:51.408654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:51.408839 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:51.408724 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 19:56:51.408839 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:51.408786 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 19:56:51.408839 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:51.408822 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:52.408800924 +0000 UTC m=+47.664032324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : configmap "kube-rbac-proxy" not found Jul 17 19:56:51.408839 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:51.408841 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:52.408831905 +0000 UTC m=+47.664063307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : secret "insights-runtime-extractor-tls" not found Jul 17 19:56:51.506021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:51.505990 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/1.log" Jul 17 19:56:51.506363 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:51.506344 2578 scope.go:117] "RemoveContainer" containerID="9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36" Jul 17 19:56:51.506528 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:51.506505 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-7grmk_openshift-console-operator(eba72d2c-0061-47b3-9339-2afcfebc950b)\"" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podUID="eba72d2c-0061-47b3-9339-2afcfebc950b" Jul 17 19:56:52.419847 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:52.419808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:52.420028 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:52.419863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:52.420028 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:52.419938 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 19:56:52.420028 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:52.419946 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 19:56:52.420133 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:52.420030 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:54.420008446 +0000 UTC m=+49.675239848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : configmap "kube-rbac-proxy" not found Jul 17 19:56:52.420133 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:52.420058 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:54.420047697 +0000 UTC m=+49.675279096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : secret "insights-runtime-extractor-tls" not found Jul 17 19:56:52.700990 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:52.700919 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jftrg_67eb9374-9fb7-4b2f-8b35-eee4d7649140/dns-node-resolver/0.log" Jul 17 19:56:53.496649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:53.496602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cq8td_a0f3d694-caca-4f28-b36a-0d7e36afb234/node-ca/0.log" Jul 17 19:56:54.439451 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:54.439409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:54.439984 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:54.439543 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:54.439984 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:54.439583 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 19:56:54.439984 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:54.439605 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 19:56:54.439984 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:54.439677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:58.43966358 +0000 UTC m=+53.694894975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : configmap "kube-rbac-proxy" not found Jul 17 19:56:54.439984 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:54.439691 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:56:58.439685343 +0000 UTC m=+53.694916738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : secret "insights-runtime-extractor-tls" not found Jul 17 19:56:54.505498 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:54.505470 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zsckk_34abb3e7-a612-4773-9220-b47320b4ce85/serve-healthcheck-canary/0.log" Jul 17 19:56:55.248505 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.248455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.248530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.248555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.248581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248689 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248708 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575c766bdf-4jwx6: secret "image-registry-tls" not found Jul 17 19:56:55.248728 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248717 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.248695139 +0000 UTC m=+66.503926537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : configmap references non-existent config key: service-ca.crt Jul 17 19:56:55.249010 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248771 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls podName:519aef38-8c7a-4de4-8224-6274fb384ddf nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.248754017 +0000 UTC m=+66.503985428 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls") pod "image-registry-575c766bdf-4jwx6" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf") : secret "image-registry-tls" not found Jul 17 19:56:55.249010 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248781 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 19:56:55.249010 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248831 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls podName:3ef3b898-af9f-41a1-8139-39e429a12416 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.248818934 +0000 UTC m=+66.504050334 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dcjw9" (UID: "3ef3b898-af9f-41a1-8139-39e429a12416") : secret "samples-operator-tls" not found Jul 17 19:56:55.249010 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248834 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 19:56:55.249010 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.248868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs podName:a27a74c6-a582-4d59-b86c-b3b6e9e3e376 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.248857289 +0000 UTC m=+66.504088691 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs") pod "router-default-765ff78b4-jfc5r" (UID: "a27a74c6-a582-4d59-b86c-b3b6e9e3e376") : secret "router-metrics-certs-default" not found Jul 17 19:56:55.349968 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.349926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:56:55.350114 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.350009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:56:55.350114 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:55.350065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:56:55.350114 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350086 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 19:56:55.350274 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350142 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 19:56:55.350274 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350172 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert podName:26b6965a-9faa-468e-9d98-0c63b8126a86 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.350152489 +0000 UTC m=+66.605383890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-97xw4" (UID: "26b6965a-9faa-468e-9d98-0c63b8126a86") : secret "networking-console-plugin-cert" not found Jul 17 19:56:55.350274 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350207 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls podName:7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.350192452 +0000 UTC m=+66.605423847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls") pod "dns-default-8pkg4" (UID: "7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8") : secret "dns-default-metrics-tls" not found Jul 17 19:56:55.350274 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350146 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:55.350274 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:55.350239 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls podName:c014a045-d831-4d06-b875-f4460634be03 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:11.350232293 +0000 UTC m=+66.605463688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xhr48" (UID: "c014a045-d831-4d06-b875-f4460634be03") : secret "cluster-monitoring-operator-tls" not found Jul 17 19:56:58.477772 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:58.477733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:58.478197 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:58.477787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:58.478197 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:58.477887 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 19:56:58.478197 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:56:58.477949 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls podName:7ff80eb7-0b29-44a8-ab00-501321c6542e nodeName:}" failed. No retries permitted until 2026-07-17 19:57:06.477930672 +0000 UTC m=+61.733162068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wrtl" (UID: "7ff80eb7-0b29-44a8-ab00-501321c6542e") : secret "insights-runtime-extractor-tls" not found Jul 17 19:56:58.478449 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:58.478430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ff80eb7-0b29-44a8-ab00-501321c6542e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:56:59.713897 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:59.713836 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:59.714281 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:59.713942 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:56:59.714281 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:56:59.714269 2578 scope.go:117] "RemoveContainer" containerID="9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36" Jul 17 19:57:00.534696 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.534675 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 19:57:00.535170 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.535156 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/1.log" Jul 17 19:57:00.535253 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.535186 2578 generic.go:356] "Generic (PLEG): container finished" podID="eba72d2c-0061-47b3-9339-2afcfebc950b" containerID="780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5" exitCode=255 Jul 17 19:57:00.535253 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.535222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" event={"ID":"eba72d2c-0061-47b3-9339-2afcfebc950b","Type":"ContainerDied","Data":"780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5"} Jul 17 19:57:00.535318 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.535267 2578 scope.go:117] "RemoveContainer" containerID="9796fec7066b1b9aad8c3da9f2536b8104039b1101a89c48a11a32723c04ac36" Jul 17 19:57:00.535556 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:00.535538 2578 scope.go:117] "RemoveContainer" containerID="780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5" Jul 17 19:57:00.535766 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:00.535744 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-7grmk_openshift-console-operator(eba72d2c-0061-47b3-9339-2afcfebc950b)\"" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podUID="eba72d2c-0061-47b3-9339-2afcfebc950b" Jul 17 19:57:01.539866 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:01.539841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 19:57:05.412068 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:05.412039 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dzmx" Jul 17 19:57:06.558846 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:06.558812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:57:06.561101 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:06.561077 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ff80eb7-0b29-44a8-ab00-501321c6542e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wrtl\" (UID: \"7ff80eb7-0b29-44a8-ab00-501321c6542e\") " pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:57:06.675319 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:06.675288 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nmtpw\"" Jul 17 19:57:06.684499 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:06.684480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wrtl" Jul 17 19:57:06.810964 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:06.810941 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wrtl"] Jul 17 19:57:06.813245 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:06.813216 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff80eb7_0b29_44a8_ab00_501321c6542e.slice/crio-04adc9dded7bb0ecb84ee327bc6111ef4e066e1f1a3882a14a129ea194d15998 WatchSource:0}: Error finding container 04adc9dded7bb0ecb84ee327bc6111ef4e066e1f1a3882a14a129ea194d15998: Status 404 returned error can't find the container with id 04adc9dded7bb0ecb84ee327bc6111ef4e066e1f1a3882a14a129ea194d15998 Jul 17 19:57:07.560119 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:07.560087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wrtl" event={"ID":"7ff80eb7-0b29-44a8-ab00-501321c6542e","Type":"ContainerStarted","Data":"04adc9dded7bb0ecb84ee327bc6111ef4e066e1f1a3882a14a129ea194d15998"} Jul 17 19:57:09.713258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:09.713223 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:57:09.713693 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:09.713320 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:57:09.713840 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:09.713825 2578 scope.go:117] "RemoveContainer" containerID="780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5" Jul 17 19:57:09.714041 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:09.714017 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-7grmk_openshift-console-operator(eba72d2c-0061-47b3-9339-2afcfebc950b)\"" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podUID="eba72d2c-0061-47b3-9339-2afcfebc950b" Jul 17 19:57:10.568871 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:10.568844 2578 scope.go:117] "RemoveContainer" containerID="780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5" Jul 17 19:57:10.569041 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:10.569023 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-7grmk_openshift-console-operator(eba72d2c-0061-47b3-9339-2afcfebc950b)\"" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podUID="eba72d2c-0061-47b3-9339-2afcfebc950b" Jul 17 19:57:11.005516 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.005481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:57:11.006004 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.005537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:57:11.006004 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.005574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:57:11.008097 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.007958 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 19:57:11.008097 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.007982 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 19:57:11.008382 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.008355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/bdddb632-f19c-48ad-83a9-7ca487182630-kube-api-access-6npxv\") pod \"network-check-target-srdd9\" (UID: \"bdddb632-f19c-48ad-83a9-7ca487182630\") " pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:57:11.018134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.018097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ecb91-80ae-4221-9797-50fa1be7eda1-metrics-certs\") pod \"network-metrics-daemon-8gscn\" (UID: \"1d6ecb91-80ae-4221-9797-50fa1be7eda1\") " pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:57:11.018134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.018127 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/908d7010-dec8-4574-9ce4-29a5d5327bec-original-pull-secret\") pod \"global-pull-secret-syncer-rqfrh\" (UID: \"908d7010-dec8-4574-9ce4-29a5d5327bec\") " pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:57:11.162118 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.162092 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clqd4\"" Jul 17 19:57:11.167019 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.166989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqfrh" Jul 17 19:57:11.170819 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.170800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:57:11.173887 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.173872 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdp2d\"" Jul 17 19:57:11.182692 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.182673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gscn" Jul 17 19:57:11.308694 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.308592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:57:11.308694 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.308685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:11.308928 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.308712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:57:11.308928 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.308736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:11.309480 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.309458 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-service-ca-bundle\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:11.311584 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.311559 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"image-registry-575c766bdf-4jwx6\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:57:11.311716 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.311604 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ef3b898-af9f-41a1-8139-39e429a12416-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dcjw9\" (UID: \"3ef3b898-af9f-41a1-8139-39e429a12416\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:57:11.311716 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.311700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27a74c6-a582-4d59-b86c-b3b6e9e3e376-metrics-certs\") pod \"router-default-765ff78b4-jfc5r\" (UID: \"a27a74c6-a582-4d59-b86c-b3b6e9e3e376\") " pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:11.409867 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.409828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:57:11.410039 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.409883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:57:11.410039 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.409915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:57:11.412747 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.412725 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8-metrics-tls\") pod \"dns-default-8pkg4\" (UID: \"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8\") " pod="openshift-dns/dns-default-8pkg4" Jul 17 19:57:11.412869 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.412806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c014a045-d831-4d06-b875-f4460634be03-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xhr48\" (UID: \"c014a045-d831-4d06-b875-f4460634be03\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:57:11.412869 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.412842 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26b6965a-9faa-468e-9d98-0c63b8126a86-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-97xw4\" (UID: \"26b6965a-9faa-468e-9d98-0c63b8126a86\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:57:11.475209 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.475177 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wqfls\"" Jul 17 19:57:11.483177 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.483146 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:57:11.524299 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.524266 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-98hb8\"" Jul 17 19:57:11.532425 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.532380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" Jul 17 19:57:11.542129 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.542100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qmmcs\"" Jul 17 19:57:11.549491 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.549273 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hqsst\"" Jul 17 19:57:11.551036 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.550844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:11.557798 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.557545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" Jul 17 19:57:11.563548 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.563337 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkppg\"" Jul 17 19:57:11.576202 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.574227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8pkg4" Jul 17 19:57:11.576202 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.574825 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5rcld\"" Jul 17 19:57:11.583371 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.583175 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" Jul 17 19:57:11.694460 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.693070 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8gscn"] Jul 17 19:57:11.740434 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.740402 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqfrh"] Jul 17 19:57:11.846873 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.846797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48"] Jul 17 19:57:11.849787 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:11.849760 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc014a045_d831_4d06_b875_f4460634be03.slice/crio-aa42c07420cbb762454c1b1ac45e0b97d8e7d3b9a3ef44e6589e43f3970e217c WatchSource:0}: Error finding container aa42c07420cbb762454c1b1ac45e0b97d8e7d3b9a3ef44e6589e43f3970e217c: Status 404 returned error can't find the container with id aa42c07420cbb762454c1b1ac45e0b97d8e7d3b9a3ef44e6589e43f3970e217c Jul 17 19:57:11.930581 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.930542 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-srdd9"] Jul 17 19:57:11.932831 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:11.932807 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:57:11.933916 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:11.933895 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdddb632_f19c_48ad_83a9_7ca487182630.slice/crio-b9abc918fea7c1ea96b53e4b7923a66b9021a91309dcd222213f499b2e5d6193 WatchSource:0}: Error finding container b9abc918fea7c1ea96b53e4b7923a66b9021a91309dcd222213f499b2e5d6193: Status 404 returned error can't find the container with id b9abc918fea7c1ea96b53e4b7923a66b9021a91309dcd222213f499b2e5d6193 Jul 17 19:57:11.936583 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:11.936549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519aef38_8c7a_4de4_8224_6274fb384ddf.slice/crio-6615ce257a05887f17c2da5033fff8c288f9dfc58566078a1550e8d5b9d67bc5 WatchSource:0}: Error finding container 6615ce257a05887f17c2da5033fff8c288f9dfc58566078a1550e8d5b9d67bc5: Status 404 returned error can't find the container with id 6615ce257a05887f17c2da5033fff8c288f9dfc58566078a1550e8d5b9d67bc5 Jul 17 19:57:12.075654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.075600 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9"] Jul 17 19:57:12.076918 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.076892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-765ff78b4-jfc5r"] Jul 17 19:57:12.078951 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.078889 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4"] Jul 17 19:57:12.080226 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.080120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8pkg4"] Jul 17 19:57:12.082011 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:12.081986 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b6965a_9faa_468e_9d98_0c63b8126a86.slice/crio-926316de403a522b1d15f9fc16de340160da5119e3b97ba97c02692cf1567ca8 WatchSource:0}: Error finding container 926316de403a522b1d15f9fc16de340160da5119e3b97ba97c02692cf1567ca8: Status 404 returned error can't find the container with id 926316de403a522b1d15f9fc16de340160da5119e3b97ba97c02692cf1567ca8 Jul 17 19:57:12.083132 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:12.083096 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee5ac9d_d9b8_416a_a2d1_f47e3cc136c8.slice/crio-659fab8e7a5178fcc2d58a1ce9518c2281c926a9d811779e1b63927661148662 WatchSource:0}: Error finding container 659fab8e7a5178fcc2d58a1ce9518c2281c926a9d811779e1b63927661148662: Status 404 returned error can't find the container with id 659fab8e7a5178fcc2d58a1ce9518c2281c926a9d811779e1b63927661148662 Jul 17 19:57:12.599792 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.599753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gscn" event={"ID":"1d6ecb91-80ae-4221-9797-50fa1be7eda1","Type":"ContainerStarted","Data":"2159eaf8ec43cc34ff14064107e2b932d52825b9c399db53bf47543a40b9eb64"} Jul 17 19:57:12.608008 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.607973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8pkg4" event={"ID":"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8","Type":"ContainerStarted","Data":"659fab8e7a5178fcc2d58a1ce9518c2281c926a9d811779e1b63927661148662"} Jul 17 19:57:12.611304 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.611274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wrtl" event={"ID":"7ff80eb7-0b29-44a8-ab00-501321c6542e","Type":"ContainerStarted","Data":"8b616c997d690b1d8cfe7d70e8799db2638b79322c27b319bb2011ae61ee7d93"} Jul 17 19:57:12.619252 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.619219 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" event={"ID":"26b6965a-9faa-468e-9d98-0c63b8126a86","Type":"ContainerStarted","Data":"926316de403a522b1d15f9fc16de340160da5119e3b97ba97c02692cf1567ca8"} Jul 17 19:57:12.623464 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.622568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-765ff78b4-jfc5r" event={"ID":"a27a74c6-a582-4d59-b86c-b3b6e9e3e376","Type":"ContainerStarted","Data":"0727010891d8ed88a664d80d90ab06e090b7a2be92cf134dad6cb838ee9d066d"} Jul 17 19:57:12.623464 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.622635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-765ff78b4-jfc5r" event={"ID":"a27a74c6-a582-4d59-b86c-b3b6e9e3e376","Type":"ContainerStarted","Data":"535ba2b89ecc8f1c9dcaaa5a7f7fadb33f4ab6ba6a46b0f5aeb722674a50444a"} Jul 17 19:57:12.630284 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.630206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" event={"ID":"3ef3b898-af9f-41a1-8139-39e429a12416","Type":"ContainerStarted","Data":"add334c154237d85346a47f8d0cc700095d31ffd4bceb3e3b1be330b43a7186d"} Jul 17 19:57:12.634689 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.634593 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" event={"ID":"c014a045-d831-4d06-b875-f4460634be03","Type":"ContainerStarted","Data":"aa42c07420cbb762454c1b1ac45e0b97d8e7d3b9a3ef44e6589e43f3970e217c"} Jul 17 19:57:12.637234 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.637163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqfrh" event={"ID":"908d7010-dec8-4574-9ce4-29a5d5327bec","Type":"ContainerStarted","Data":"37d5c4c8b517c6768d7c50873400fafe84b28b8c7b909deb8e3bd07854acf55b"} Jul 17 19:57:12.641656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.641019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" event={"ID":"519aef38-8c7a-4de4-8224-6274fb384ddf","Type":"ContainerStarted","Data":"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8"} Jul 17 19:57:12.641656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.641070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" event={"ID":"519aef38-8c7a-4de4-8224-6274fb384ddf","Type":"ContainerStarted","Data":"6615ce257a05887f17c2da5033fff8c288f9dfc58566078a1550e8d5b9d67bc5"} Jul 17 19:57:12.641996 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.641947 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:57:12.648541 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.648273 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-765ff78b4-jfc5r" podStartSLOduration=63.648255799 podStartE2EDuration="1m3.648255799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:12.643674388 +0000 UTC m=+67.898905807" watchObservedRunningTime="2026-07-17 19:57:12.648255799 +0000 UTC m=+67.903487218" Jul 17 19:57:12.664092 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.663222 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" podStartSLOduration=66.663197696 podStartE2EDuration="1m6.663197696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:12.661830786 +0000 UTC m=+67.917062204" watchObservedRunningTime="2026-07-17 19:57:12.663197696 +0000 UTC m=+67.918429095" Jul 17 19:57:12.671292 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.670439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-srdd9" event={"ID":"bdddb632-f19c-48ad-83a9-7ca487182630","Type":"ContainerStarted","Data":"c37dede1cb15feeeebc34a73c11a2e49fb3cadb6b3f6b6f92509f65ab2305506"} Jul 17 19:57:12.671292 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.670498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-srdd9" event={"ID":"bdddb632-f19c-48ad-83a9-7ca487182630","Type":"ContainerStarted","Data":"b9abc918fea7c1ea96b53e4b7923a66b9021a91309dcd222213f499b2e5d6193"} Jul 17 19:57:12.671292 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.671235 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:57:12.686853 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:12.686073 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-srdd9" podStartSLOduration=67.686049608 podStartE2EDuration="1m7.686049608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:12.684915643 +0000 UTC m=+67.940147061" watchObservedRunningTime="2026-07-17 19:57:12.686049608 +0000 UTC m=+67.941281027" Jul 17 19:57:13.551869 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:13.551835 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:13.555368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:13.555337 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:13.677404 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:13.677188 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:13.679168 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:13.678984 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-765ff78b4-jfc5r" Jul 17 19:57:19.701224 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.701130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" event={"ID":"c014a045-d831-4d06-b875-f4460634be03","Type":"ContainerStarted","Data":"dca575b17804dc600af2fea9e4ed2cae64501db2c891336b870c73f17c4d3054"} Jul 17 19:57:19.702694 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.702669 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqfrh" event={"ID":"908d7010-dec8-4574-9ce4-29a5d5327bec","Type":"ContainerStarted","Data":"dd6a8689b9bc110e71fa41b473abd6e0e7f217542f99f8d27584d14fd04ae547"} Jul 17 19:57:19.704434 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.704401 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gscn" event={"ID":"1d6ecb91-80ae-4221-9797-50fa1be7eda1","Type":"ContainerStarted","Data":"106cce54b4a8a3cb03ec6fb9ce8d6ace66c45f9d96b1fb038e9e8b9edcfa03ed"} Jul 17 19:57:19.704561 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.704454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gscn" event={"ID":"1d6ecb91-80ae-4221-9797-50fa1be7eda1","Type":"ContainerStarted","Data":"3179def39486908fb526c0a59357f3bc2f1e2e7076f4f442d4daa62c16217a75"} Jul 17 19:57:19.706206 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.706180 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8pkg4" event={"ID":"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8","Type":"ContainerStarted","Data":"71af68df3c334738b50ba099edf5d58ff3d1ba76a761968fceff071c97adff1a"} Jul 17 19:57:19.706317 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.706229 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8pkg4" event={"ID":"7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8","Type":"ContainerStarted","Data":"0bb49c4720c66ba268f7f5e1846ed8c3f34d05de549c7939a4aa19c897acd856"} Jul 17 19:57:19.706317 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.706259 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8pkg4" Jul 17 19:57:19.707824 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.707803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wrtl" event={"ID":"7ff80eb7-0b29-44a8-ab00-501321c6542e","Type":"ContainerStarted","Data":"da281ea9dec705268944043e482e9f707d86d210bdc5b5db0da023feb54ef8a3"} Jul 17 19:57:19.709069 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.709052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" event={"ID":"26b6965a-9faa-468e-9d98-0c63b8126a86","Type":"ContainerStarted","Data":"b8c6bde600a2b03ce2bf3b38f6d7f5046bec7273a10c3b339104b50adba0bfdb"} Jul 17 19:57:19.710738 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.710721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" event={"ID":"3ef3b898-af9f-41a1-8139-39e429a12416","Type":"ContainerStarted","Data":"a179b81b71df1b49cdae07cfeee36ac98cbd3c83a8d58d7672cac43d46f90b46"} Jul 17 19:57:19.710830 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.710753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" event={"ID":"3ef3b898-af9f-41a1-8139-39e429a12416","Type":"ContainerStarted","Data":"fffb59b6047ba38ab29eb30c6d9ecd4041d18e5ecfefcaef4c6ffd95c287e1d3"} Jul 17 19:57:19.720099 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.720058 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xhr48" podStartSLOduration=63.399683872 podStartE2EDuration="1m10.720041231s" totalImagesPullingTime="7.320357359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.719364946 +0000 UTC m=+74.974596361" watchObservedRunningTime="2026-07-17 19:57:19.720041231 +0000 UTC m=+74.975272650" Jul 17 19:57:19.735765 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.735722 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-97xw4" podStartSLOduration=39.648986585 podStartE2EDuration="46.735706363s" totalImagesPullingTime="7.086719778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.734895091 +0000 UTC m=+74.990126509" watchObservedRunningTime="2026-07-17 19:57:19.735706363 +0000 UTC m=+74.990937833" Jul 17 19:57:19.755976 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.755917 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8pkg4" podStartSLOduration=33.670606376 podStartE2EDuration="40.755893194s" totalImagesPullingTime="7.085286818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.754364207 +0000 UTC m=+75.009595625" watchObservedRunningTime="2026-07-17 19:57:19.755893194 +0000 UTC m=+75.011124612" Jul 17 19:57:19.771812 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.771747 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8gscn" podStartSLOduration=68.017375076 podStartE2EDuration="1m14.771730235s" totalImagesPullingTime="6.754355159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.77039922 +0000 UTC m=+75.025630638" watchObservedRunningTime="2026-07-17 19:57:19.771730235 +0000 UTC m=+75.026961653" Jul 17 19:57:19.809988 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.809930 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rqfrh" podStartSLOduration=66.394936337 podStartE2EDuration="1m13.809875624s" totalImagesPullingTime="7.414939287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.787320768 +0000 UTC m=+75.042552186" watchObservedRunningTime="2026-07-17 19:57:19.809875624 +0000 UTC m=+75.065107043" Jul 17 19:57:19.811326 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:19.811282 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dcjw9" podStartSLOduration=63.793069158 podStartE2EDuration="1m10.811266256s" totalImagesPullingTime="7.018197098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:19.809823347 +0000 UTC m=+75.065054764" watchObservedRunningTime="2026-07-17 19:57:19.811266256 +0000 UTC m=+75.066497674" Jul 17 19:57:23.734672 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:23.734625 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wrtl" event={"ID":"7ff80eb7-0b29-44a8-ab00-501321c6542e","Type":"ContainerStarted","Data":"5f40fff394be67ca1401207c6b1bdabb262e31aa213ee8fe611f95d45d50bd58"} Jul 17 19:57:23.752249 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:23.752201 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2wrtl" podStartSLOduration=17.332248186 podStartE2EDuration="33.752185499s" totalImagesPullingTime="16.419937313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:56:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:23.751294901 +0000 UTC m=+79.006526319" watchObservedRunningTime="2026-07-17 19:57:23.752185499 +0000 UTC m=+79.007416915" Jul 17 19:57:25.249818 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.249785 2578 scope.go:117] "RemoveContainer" containerID="780d55c4db40691c1571d2671d1f96ae9f5278f9fcbdf65ad43337a3b9fd5aa5" Jul 17 19:57:25.743087 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.743062 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 19:57:25.743254 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.743119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" event={"ID":"eba72d2c-0061-47b3-9339-2afcfebc950b","Type":"ContainerStarted","Data":"18536295c2e75ce04320170d44c97c135ac6f08748b84c0c06febf070193bed4"} Jul 17 19:57:25.743387 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.743369 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:57:25.755851 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.755826 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" Jul 17 19:57:25.758696 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:25.758634 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-7grmk" podStartSLOduration=67.729477567 podStartE2EDuration="1m16.75860432s" totalImagesPullingTime="9.029126753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:25.757954703 +0000 UTC m=+81.013186132" watchObservedRunningTime="2026-07-17 19:57:25.75860432 +0000 UTC m=+81.013835736" Jul 17 19:57:29.717447 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:29.717337 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8pkg4" Jul 17 19:57:32.297433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.297400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-v7phx"] Jul 17 19:57:32.300112 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.300094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.302289 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.302258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 17 19:57:32.303422 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.303402 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 17 19:57:32.304371 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.304351 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-smzhh\"" Jul 17 19:57:32.304371 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.304363 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 17 19:57:32.304503 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.304414 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 19:57:32.310370 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.310351 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-v7phx"] Jul 17 19:57:32.314686 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.314659 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pwdmd"] Jul 17 19:57:32.316724 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.316703 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.318840 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.318823 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 19:57:32.318993 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.318973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bmrll\"" Jul 17 19:57:32.319085 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.319051 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 19:57:32.319159 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.319141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 19:57:32.376959 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.376923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-sys\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.376959 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.376961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-textfile\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.376988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8ms\" (UniqueName: \"kubernetes.io/projected/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-api-access-4m8ms\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwd5\" (UniqueName: \"kubernetes.io/projected/28c5127e-d725-412a-ac46-1597a907e696-kube-api-access-6zwd5\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae49adf0-ffcb-46bf-828c-fcc66b70a542-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-wtmp\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-metrics-client-ca\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.377433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.377433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.377433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-root\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.377433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.377283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.477910 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.477860 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-root\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.477910 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.477907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.477980 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-root\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-sys\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-textfile\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8ms\" (UniqueName: \"kubernetes.io/projected/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-api-access-4m8ms\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478114 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-sys\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478134 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478116 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwd5\" (UniqueName: \"kubernetes.io/projected/28c5127e-d725-412a-ac46-1597a907e696-kube-api-access-6zwd5\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae49adf0-ffcb-46bf-828c-fcc66b70a542-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:32.478219 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-wtmp\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:32.478283 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls podName:28c5127e-d725-412a-ac46-1597a907e696 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:32.978261046 +0000 UTC m=+88.233492445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls") pod "node-exporter-pwdmd" (UID: "28c5127e-d725-412a-ac46-1597a907e696") : secret "node-exporter-tls" not found Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-metrics-client-ca\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478381 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478401 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-textfile\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:32.478587 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:32.478657 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls podName:ae49adf0-ffcb-46bf-828c-fcc66b70a542 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:32.978641122 +0000 UTC m=+88.233872530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-v7phx" (UID: "ae49adf0-ffcb-46bf-828c-fcc66b70a542") : secret "kube-state-metrics-tls" not found Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478660 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae49adf0-ffcb-46bf-828c-fcc66b70a542-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478730 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-wtmp\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.478922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.478844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.479323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.479090 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-metrics-client-ca\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.479323 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.479170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.479575 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.479549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.480689 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.480671 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.480839 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.480822 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.486347 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.486325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwd5\" (UniqueName: \"kubernetes.io/projected/28c5127e-d725-412a-ac46-1597a907e696-kube-api-access-6zwd5\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.486803 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.486784 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8ms\" (UniqueName: \"kubernetes.io/projected/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-api-access-4m8ms\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.984333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.984300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.984520 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.984354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:32.986698 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.986666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28c5127e-d725-412a-ac46-1597a907e696-node-exporter-tls\") pod \"node-exporter-pwdmd\" (UID: \"28c5127e-d725-412a-ac46-1597a907e696\") " pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:32.986790 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:32.986698 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae49adf0-ffcb-46bf-828c-fcc66b70a542-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-v7phx\" (UID: \"ae49adf0-ffcb-46bf-828c-fcc66b70a542\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:33.209258 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.209219 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" Jul 17 19:57:33.225300 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.225256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwdmd" Jul 17 19:57:33.236824 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:33.236740 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c5127e_d725_412a_ac46_1597a907e696.slice/crio-c00eb45a6b3c7c43fbe378b7176b43d5c1484414b63838e174133456dd783c9d WatchSource:0}: Error finding container c00eb45a6b3c7c43fbe378b7176b43d5c1484414b63838e174133456dd783c9d: Status 404 returned error can't find the container with id c00eb45a6b3c7c43fbe378b7176b43d5c1484414b63838e174133456dd783c9d Jul 17 19:57:33.341157 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.341125 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:57:33.348218 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.348191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.350370 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350349 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 19:57:33.350632 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350602 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d4xzg\"" Jul 17 19:57:33.350723 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350676 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 19:57:33.350723 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350688 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 19:57:33.350859 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350621 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 19:57:33.350859 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350858 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 19:57:33.351021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 19:57:33.351021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350938 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 19:57:33.351021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350963 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 19:57:33.351021 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.350923 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 19:57:33.354649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.354629 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-v7phx"] Jul 17 19:57:33.358080 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:33.357811 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae49adf0_ffcb_46bf_828c_fcc66b70a542.slice/crio-bf7b1d6094fefd7d1ff63f41bb8b2d85bea1af50db8a1f2ac9589e6c5071dde1 WatchSource:0}: Error finding container bf7b1d6094fefd7d1ff63f41bb8b2d85bea1af50db8a1f2ac9589e6c5071dde1: Status 404 returned error can't find the container with id bf7b1d6094fefd7d1ff63f41bb8b2d85bea1af50db8a1f2ac9589e6c5071dde1 Jul 17 19:57:33.358080 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.357877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:57:33.387203 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wsm\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387773 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.387773 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.387668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.488929 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.488833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.488929 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.488881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.488929 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.488915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.488957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.488982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:33.489046 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle podName:bd73de37-1ba8-4110-a18f-f36a05d53a7e nodeName:}" failed. No retries permitted until 2026-07-17 19:57:33.989020719 +0000 UTC m=+89.244252122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e") : configmap references non-existent config key: ca-bundle.crt Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wsm\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489468 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489468 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489468 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489468 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.489468 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.489330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.490574 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.490544 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.492132 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.492068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.492293 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:33.492169 2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 17 19:57:33.492293 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:33.492220 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls podName:bd73de37-1ba8-4110-a18f-f36a05d53a7e nodeName:}" failed. No retries permitted until 2026-07-17 19:57:33.992202875 +0000 UTC m=+89.247434272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e") : secret "alertmanager-main-tls" not found Jul 17 19:57:33.492293 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.492231 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.492927 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.492566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.492927 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.492597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.493181 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.493149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.493260 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.493191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.493313 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.493292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.494310 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.494286 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.494835 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.494814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.499338 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.499318 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wsm\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.771800 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.771709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" event={"ID":"ae49adf0-ffcb-46bf-828c-fcc66b70a542","Type":"ContainerStarted","Data":"bf7b1d6094fefd7d1ff63f41bb8b2d85bea1af50db8a1f2ac9589e6c5071dde1"} Jul 17 19:57:33.772911 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.772877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdmd" event={"ID":"28c5127e-d725-412a-ac46-1597a907e696","Type":"ContainerStarted","Data":"c00eb45a6b3c7c43fbe378b7176b43d5c1484414b63838e174133456dd783c9d"} Jul 17 19:57:33.993493 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.993463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.993790 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.993535 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.994525 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.994499 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:33.996344 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:33.996324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:34.260852 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.260812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:57:34.444433 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.444408 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:57:34.447022 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:34.446997 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd73de37_1ba8_4110_a18f_f36a05d53a7e.slice/crio-d06ac1aed5c3279177575a74be6d3edbbb274820c328c531b4c37354e063d595 WatchSource:0}: Error finding container d06ac1aed5c3279177575a74be6d3edbbb274820c328c531b4c37354e063d595: Status 404 returned error can't find the container with id d06ac1aed5c3279177575a74be6d3edbbb274820c328c531b4c37354e063d595 Jul 17 19:57:34.685902 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.685874 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:57:34.779039 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.778999 2578 generic.go:356] "Generic (PLEG): container finished" podID="28c5127e-d725-412a-ac46-1597a907e696" containerID="5fa6d6492bdba8e5672658d4118091f8089e847cef148052a70f21b36a5b9259" exitCode=0 Jul 17 19:57:34.779218 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.779087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdmd" event={"ID":"28c5127e-d725-412a-ac46-1597a907e696","Type":"ContainerDied","Data":"5fa6d6492bdba8e5672658d4118091f8089e847cef148052a70f21b36a5b9259"} Jul 17 19:57:34.780599 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:34.780577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"d06ac1aed5c3279177575a74be6d3edbbb274820c328c531b4c37354e063d595"} Jul 17 19:57:35.787393 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.787360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdmd" event={"ID":"28c5127e-d725-412a-ac46-1597a907e696","Type":"ContainerStarted","Data":"6bb06910636748ec2d8148b761d47d33160bdcfe8254c4ac0452da61d0e54ac4"} Jul 17 19:57:35.787768 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.787423 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdmd" event={"ID":"28c5127e-d725-412a-ac46-1597a907e696","Type":"ContainerStarted","Data":"198576da1a6246cea48d1d6908e51f84e6f188db46c0038cc1a667d88a686fac"} Jul 17 19:57:35.789632 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.789594 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" event={"ID":"ae49adf0-ffcb-46bf-828c-fcc66b70a542","Type":"ContainerStarted","Data":"c1ece4757bc14781e982dd81aa2e3ee39a379b4a74d6d3a252818db81969286d"} Jul 17 19:57:35.789715 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.789661 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" event={"ID":"ae49adf0-ffcb-46bf-828c-fcc66b70a542","Type":"ContainerStarted","Data":"9eb9f1f130ff34a7fd72a5688bef7a6a642c6405601f225dd03929c01e79aecb"} Jul 17 19:57:35.789715 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.789688 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" event={"ID":"ae49adf0-ffcb-46bf-828c-fcc66b70a542","Type":"ContainerStarted","Data":"bd8a55a847baf85fa23fc0d6fd4c891ee820401f5f92662b517d2ed94ba63ae3"} Jul 17 19:57:35.806578 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.806531 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pwdmd" podStartSLOduration=2.6547869840000002 podStartE2EDuration="3.806515877s" totalImagesPullingTime="1.079031578s" totalInitContainerRuntime="72.697315ms" isStatefulPod=true podCreationTimestamp="2026-07-17 19:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:35.805458521 +0000 UTC m=+91.060689940" watchObservedRunningTime="2026-07-17 19:57:35.806515877 +0000 UTC m=+91.061747293" Jul 17 19:57:35.821261 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:35.821113 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-v7phx" podStartSLOduration=2.179572213 podStartE2EDuration="3.821090775s" totalImagesPullingTime="1.641518562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:35.820498423 +0000 UTC m=+91.075729871" watchObservedRunningTime="2026-07-17 19:57:35.821090775 +0000 UTC m=+91.076322196" Jul 17 19:57:36.795030 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:36.794992 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777" exitCode=0 Jul 17 19:57:36.795399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:36.795073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777"} Jul 17 19:57:37.072462 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.072381 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544"] Jul 17 19:57:37.074511 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.074492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.076879 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.076851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 17 19:57:37.076965 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.076860 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dw8vk\"" Jul 17 19:57:37.084191 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.084171 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544"] Jul 17 19:57:37.119921 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.119885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkhbh\" (UniqueName: \"kubernetes.io/projected/dda56631-feca-4d27-8fc5-63ddd1719c00-kube-api-access-bkhbh\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.120089 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.119986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.190474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.189308 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:57:37.220588 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.220557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.220761 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.220600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkhbh\" (UniqueName: \"kubernetes.io/projected/dda56631-feca-4d27-8fc5-63ddd1719c00-kube-api-access-bkhbh\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.220761 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:37.220716 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 17 19:57:37.220831 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:57:37.220823 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert podName:dda56631-feca-4d27-8fc5-63ddd1719c00 nodeName:}" failed. No retries permitted until 2026-07-17 19:57:37.72080412 +0000 UTC m=+92.976035529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-8x544" (UID: "dda56631-feca-4d27-8fc5-63ddd1719c00") : secret "monitoring-plugin-cert" not found Jul 17 19:57:37.228990 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.228963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkhbh\" (UniqueName: \"kubernetes.io/projected/dda56631-feca-4d27-8fc5-63ddd1719c00-kube-api-access-bkhbh\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.724919 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.724880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.727274 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.727247 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dda56631-feca-4d27-8fc5-63ddd1719c00-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8x544\" (UID: \"dda56631-feca-4d27-8fc5-63ddd1719c00\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:37.983558 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:37.983462 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:38.413652 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.413602 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544"] Jul 17 19:57:38.416492 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:38.416465 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda56631_feca_4d27_8fc5_63ddd1719c00.slice/crio-56a951d4b89b2a472de32cbcbc45ff68ab13eabf00ff396e49f26139d252b6ca WatchSource:0}: Error finding container 56a951d4b89b2a472de32cbcbc45ff68ab13eabf00ff396e49f26139d252b6ca: Status 404 returned error can't find the container with id 56a951d4b89b2a472de32cbcbc45ff68ab13eabf00ff396e49f26139d252b6ca Jul 17 19:57:38.806516 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.806408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea"} Jul 17 19:57:38.806516 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.806495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b"} Jul 17 19:57:38.806763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.806539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246"} Jul 17 19:57:38.806763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.806578 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5"} Jul 17 19:57:38.806763 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.806634 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf"} Jul 17 19:57:38.807922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:38.807899 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" event={"ID":"dda56631-feca-4d27-8fc5-63ddd1719c00","Type":"ContainerStarted","Data":"56a951d4b89b2a472de32cbcbc45ff68ab13eabf00ff396e49f26139d252b6ca"} Jul 17 19:57:40.817866 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.817823 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" event={"ID":"dda56631-feca-4d27-8fc5-63ddd1719c00","Type":"ContainerStarted","Data":"4382e78f5773d239c3f4e3eac6f969b24381ac295013df898e94ca567d8ed342"} Jul 17 19:57:40.818309 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.817893 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:40.824491 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.824462 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" Jul 17 19:57:40.826333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.826299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerStarted","Data":"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a"} Jul 17 19:57:40.834809 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.834764 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8x544" podStartSLOduration=2.41243364 podStartE2EDuration="3.834750228s" totalImagesPullingTime="1.422316588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:40.833392464 +0000 UTC m=+96.088623881" watchObservedRunningTime="2026-07-17 19:57:40.834750228 +0000 UTC m=+96.089981697" Jul 17 19:57:40.875674 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:40.875600 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.857817506 podStartE2EDuration="7.875582249s" totalImagesPullingTime="4.000406914s" totalInitContainerRuntime="17.357829ms" isStatefulPod=false podCreationTimestamp="2026-07-17 19:57:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:40.873043528 +0000 UTC m=+96.128274944" watchObservedRunningTime="2026-07-17 19:57:40.875582249 +0000 UTC m=+96.130813666" Jul 17 19:57:42.646413 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.646370 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:57:42.791526 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.791482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:57:42.791704 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.791633 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.794064 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794038 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 19:57:42.794209 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794086 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qqnqn\"" Jul 17 19:57:42.794851 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794825 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 19:57:42.794976 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794866 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 19:57:42.794976 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794890 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 19:57:42.794976 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794891 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 19:57:42.794976 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 19:57:42.795175 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.794827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 19:57:42.799086 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.798903 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 19:57:42.873352 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873352 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9sjw\" (UniqueName: \"kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.873549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.873536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974467 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974467 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9sjw\" (UniqueName: \"kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974467 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974467 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974826 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974826 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.974826 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.974585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.975243 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.975219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.975243 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.975230 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.975383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.975349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.975540 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.975520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.977007 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.976990 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.977103 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.977008 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:42.981553 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:42.981534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9sjw\" (UniqueName: \"kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw\") pod \"console-5dcf74c77d-tbkmf\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:43.101835 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:43.101798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:43.267818 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:43.267746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:57:43.271355 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:57:43.271327 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705d8a7d_1f39_4efa_81c9_954861267c4d.slice/crio-245ea2635ec0b8ddea4b1b85137caf678500cf3965c80ed3946d4b98490038ec WatchSource:0}: Error finding container 245ea2635ec0b8ddea4b1b85137caf678500cf3965c80ed3946d4b98490038ec: Status 404 returned error can't find the container with id 245ea2635ec0b8ddea4b1b85137caf678500cf3965c80ed3946d4b98490038ec Jul 17 19:57:43.839549 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:43.839509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dcf74c77d-tbkmf" event={"ID":"705d8a7d-1f39-4efa-81c9-954861267c4d","Type":"ContainerStarted","Data":"245ea2635ec0b8ddea4b1b85137caf678500cf3965c80ed3946d4b98490038ec"} Jul 17 19:57:44.683544 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:44.683509 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-srdd9" Jul 17 19:57:46.852950 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:46.852916 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dcf74c77d-tbkmf" event={"ID":"705d8a7d-1f39-4efa-81c9-954861267c4d","Type":"ContainerStarted","Data":"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b"} Jul 17 19:57:46.869130 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:46.869084 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dcf74c77d-tbkmf" podStartSLOduration=2.207125575 podStartE2EDuration="4.869068012s" totalImagesPullingTime="2.661942437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:57:46.8678726 +0000 UTC m=+102.123104020" watchObservedRunningTime="2026-07-17 19:57:46.869068012 +0000 UTC m=+102.124299429" Jul 17 19:57:53.102519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:53.102483 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:53.103030 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:53.102578 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:53.107255 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:53.107234 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:57:53.882585 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:57:53.882552 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:58:00.908013 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:00.907982 2578 generic.go:356] "Generic (PLEG): container finished" podID="1be32d05-bf30-4d24-9ce0-483471ef371c" containerID="e2093f8d5f4819d0d307aab692a26d5e1cced4c9a7c10ffb93f6037cf1307033" exitCode=0 Jul 17 19:58:00.908465 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:00.908054 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" event={"ID":"1be32d05-bf30-4d24-9ce0-483471ef371c","Type":"ContainerDied","Data":"e2093f8d5f4819d0d307aab692a26d5e1cced4c9a7c10ffb93f6037cf1307033"} Jul 17 19:58:00.908465 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:00.908364 2578 scope.go:117] "RemoveContainer" containerID="e2093f8d5f4819d0d307aab692a26d5e1cced4c9a7c10ffb93f6037cf1307033" Jul 17 19:58:01.913993 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:01.913957 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-g6c7k" event={"ID":"1be32d05-bf30-4d24-9ce0-483471ef371c","Type":"ContainerStarted","Data":"104ce88d41139c98890d92d8ada700dfd6838deaa019140b450e92b740588a0d"} Jul 17 19:58:02.214498 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.214405 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" podUID="519aef38-8c7a-4de4-8224-6274fb384ddf" containerName="registry" containerID="cri-o://f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8" gracePeriod=30 Jul 17 19:58:02.457785 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.457763 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:58:02.653402 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653373 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653581 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653432 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653581 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653581 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653570 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653786 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653658 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp5jb\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653786 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653694 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653786 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653737 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.653786 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.653765 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca\") pod \"519aef38-8c7a-4de4-8224-6274fb384ddf\" (UID: \"519aef38-8c7a-4de4-8224-6274fb384ddf\") " Jul 17 19:58:02.654114 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.654079 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:58:02.654392 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.654366 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:58:02.656314 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.656283 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:02.656424 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.656373 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:02.656424 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.656372 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:58:02.656505 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.656425 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb" (OuterVolumeSpecName: "kube-api-access-qp5jb") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "kube-api-access-qp5jb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:58:02.656505 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.656449 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:58:02.664363 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.664338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "519aef38-8c7a-4de4-8224-6274fb384ddf" (UID: "519aef38-8c7a-4de4-8224-6274fb384ddf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:58:02.754568 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754534 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/519aef38-8c7a-4de4-8224-6274fb384ddf-ca-trust-extracted\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754568 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754561 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-bound-sa-token\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754568 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754570 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754568 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754578 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp5jb\" (UniqueName: \"kubernetes.io/projected/519aef38-8c7a-4de4-8224-6274fb384ddf-kube-api-access-qp5jb\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754853 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754586 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-registry-certificates\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754853 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754595 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-image-registry-private-configuration\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754853 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754604 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/519aef38-8c7a-4de4-8224-6274fb384ddf-trusted-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.754853 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.754632 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/519aef38-8c7a-4de4-8224-6274fb384ddf-installation-pull-secrets\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:02.918887 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.918798 2578 generic.go:356] "Generic (PLEG): container finished" podID="519aef38-8c7a-4de4-8224-6274fb384ddf" containerID="f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8" exitCode=0 Jul 17 19:58:02.918887 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.918844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" Jul 17 19:58:02.918887 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.918848 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" event={"ID":"519aef38-8c7a-4de4-8224-6274fb384ddf","Type":"ContainerDied","Data":"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8"} Jul 17 19:58:02.919427 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.918930 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-575c766bdf-4jwx6" event={"ID":"519aef38-8c7a-4de4-8224-6274fb384ddf","Type":"ContainerDied","Data":"6615ce257a05887f17c2da5033fff8c288f9dfc58566078a1550e8d5b9d67bc5"} Jul 17 19:58:02.919427 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.918960 2578 scope.go:117] "RemoveContainer" containerID="f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8" Jul 17 19:58:02.927970 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.927953 2578 scope.go:117] "RemoveContainer" containerID="f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8" Jul 17 19:58:02.928216 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:02.928197 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8\": container with ID starting with f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8 not found: ID does not exist" containerID="f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8" Jul 17 19:58:02.928262 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.928224 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8"} err="failed to get container status \"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8\": rpc error: code = NotFound desc = could not find container \"f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8\": container with ID starting with f2704a5f3418cfec18e367ef069378382e0f390bf57953f95bb35eeb937041f8 not found: ID does not exist" Jul 17 19:58:02.938386 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.938364 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:58:02.942368 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:02.942346 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-575c766bdf-4jwx6"] Jul 17 19:58:03.252866 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:03.252783 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="519aef38-8c7a-4de4-8224-6274fb384ddf" path="/var/lib/kubelet/pods/519aef38-8c7a-4de4-8224-6274fb384ddf/volumes" Jul 17 19:58:14.968092 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:14.968060 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c210c3d-d346-4651-904a-779637bc420b" containerID="52e408811501cc283f74200aa8dea4543bb444bc91e5b807f79b93450a349c77" exitCode=0 Jul 17 19:58:14.968489 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:14.968133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" event={"ID":"2c210c3d-d346-4651-904a-779637bc420b","Type":"ContainerDied","Data":"52e408811501cc283f74200aa8dea4543bb444bc91e5b807f79b93450a349c77"} Jul 17 19:58:14.968489 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:14.968447 2578 scope.go:117] "RemoveContainer" containerID="52e408811501cc283f74200aa8dea4543bb444bc91e5b807f79b93450a349c77" Jul 17 19:58:15.974830 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:15.974789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6mg2j" event={"ID":"2c210c3d-d346-4651-904a-779637bc420b","Type":"ContainerStarted","Data":"647955784fc38cd313d4d6f0e0dfabb7e43e65af0693c4f808d53b3d82b518f2"} Jul 17 19:58:15.976178 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:15.976153 2578 generic.go:356] "Generic (PLEG): container finished" podID="20a98849-040d-4d02-8485-70ed0281cf4f" containerID="45287d5ab5508ef4698c2cbd739a960f76c99d94637120bf114584263af8abac" exitCode=0 Jul 17 19:58:15.976314 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:15.976217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-2v946" event={"ID":"20a98849-040d-4d02-8485-70ed0281cf4f","Type":"ContainerDied","Data":"45287d5ab5508ef4698c2cbd739a960f76c99d94637120bf114584263af8abac"} Jul 17 19:58:15.976548 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:15.976533 2578 scope.go:117] "RemoveContainer" containerID="45287d5ab5508ef4698c2cbd739a960f76c99d94637120bf114584263af8abac" Jul 17 19:58:16.982787 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:16.982751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-2v946" event={"ID":"20a98849-040d-4d02-8485-70ed0281cf4f","Type":"ContainerStarted","Data":"4972f70a04366fd796e3106dc825c002c9011c1d1cec6ca545c9dd3e5016ee8f"} Jul 17 19:58:52.575899 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.575860 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:52.576478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576221 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="alertmanager" containerID="cri-o://38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" gracePeriod=120 Jul 17 19:58:52.576478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576287 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-metric" containerID="cri-o://dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" gracePeriod=120 Jul 17 19:58:52.576478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576400 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy" containerID="cri-o://460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" gracePeriod=120 Jul 17 19:58:52.576695 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576441 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="config-reloader" containerID="cri-o://5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" gracePeriod=120 Jul 17 19:58:52.576695 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576409 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="prom-label-proxy" containerID="cri-o://a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" gracePeriod=120 Jul 17 19:58:52.577220 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:52.576586 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-web" containerID="cri-o://ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" gracePeriod=120 Jul 17 19:58:53.127418 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127383 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" exitCode=0 Jul 17 19:58:53.127418 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127414 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" exitCode=0 Jul 17 19:58:53.127418 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127420 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" exitCode=0 Jul 17 19:58:53.127418 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127426 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" exitCode=0 Jul 17 19:58:53.127713 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a"} Jul 17 19:58:53.127713 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127519 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b"} Jul 17 19:58:53.127713 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5"} Jul 17 19:58:53.127713 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.127571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf"} Jul 17 19:58:53.830007 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.829985 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:53.982949 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.982911 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88wsm\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983157 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.982970 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983157 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.982997 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983276 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983155 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983276 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983184 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983276 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983201 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983276 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983250 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983283 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983320 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983358 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983359 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983398 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983478 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983463 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out\") pod \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\" (UID: \"bd73de37-1ba8-4110-a18f-f36a05d53a7e\") " Jul 17 19:58:53.983929 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.983780 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-metrics-client-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:53.986364 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986084 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm" (OuterVolumeSpecName: "kube-api-access-88wsm") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "kube-api-access-88wsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:58:53.986364 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986332 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out" (OuterVolumeSpecName: "config-out") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:58:53.986364 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986335 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:58:53.986671 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986650 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.986738 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986655 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:58:53.986738 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986706 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.986848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986751 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.986939 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.986919 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:58:53.988458 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.988430 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.988558 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.988512 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume" (OuterVolumeSpecName: "config-volume") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.990051 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.990026 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:53.996439 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:53.996415 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config" (OuterVolumeSpecName: "web-config") pod "bd73de37-1ba8-4110-a18f-f36a05d53a7e" (UID: "bd73de37-1ba8-4110-a18f-f36a05d53a7e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:58:54.084509 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084476 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88wsm\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-kube-api-access-88wsm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084509 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084504 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084509 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084514 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084524 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-web-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084532 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-main-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084541 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd73de37-1ba8-4110-a18f-f36a05d53a7e-tls-assets\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084548 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-cluster-tls-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084559 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-main-db\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084567 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-volume\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084576 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd73de37-1ba8-4110-a18f-f36a05d53a7e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084585 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd73de37-1ba8-4110-a18f-f36a05d53a7e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.084748 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.084593 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd73de37-1ba8-4110-a18f-f36a05d53a7e-config-out\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:58:54.134041 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134004 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" exitCode=0 Jul 17 19:58:54.134041 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134030 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerID="ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" exitCode=0 Jul 17 19:58:54.134236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134078 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.134236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134089 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea"} Jul 17 19:58:54.134236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246"} Jul 17 19:58:54.134236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd73de37-1ba8-4110-a18f-f36a05d53a7e","Type":"ContainerDied","Data":"d06ac1aed5c3279177575a74be6d3edbbb274820c328c531b4c37354e063d595"} Jul 17 19:58:54.134236 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.134229 2578 scope.go:117] "RemoveContainer" containerID="a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" Jul 17 19:58:54.142927 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.142909 2578 scope.go:117] "RemoveContainer" containerID="dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" Jul 17 19:58:54.152431 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.152410 2578 scope.go:117] "RemoveContainer" containerID="460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" Jul 17 19:58:54.157364 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.157339 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:54.160163 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.160144 2578 scope.go:117] "RemoveContainer" containerID="ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" Jul 17 19:58:54.163966 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.163943 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:54.167540 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.167521 2578 scope.go:117] "RemoveContainer" containerID="5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" Jul 17 19:58:54.174701 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.174681 2578 scope.go:117] "RemoveContainer" containerID="38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" Jul 17 19:58:54.182088 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.182068 2578 scope.go:117] "RemoveContainer" containerID="7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777" Jul 17 19:58:54.188474 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188451 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:54.188765 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188753 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="init-config-reloader" Jul 17 19:58:54.188808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188768 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="init-config-reloader" Jul 17 19:58:54.188808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188780 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy" Jul 17 19:58:54.188808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188786 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy" Jul 17 19:58:54.188808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188803 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-metric" Jul 17 19:58:54.188808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188809 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-metric" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188818 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519aef38-8c7a-4de4-8224-6274fb384ddf" containerName="registry" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188823 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="519aef38-8c7a-4de4-8224-6274fb384ddf" containerName="registry" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188836 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="alertmanager" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188842 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="alertmanager" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188852 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="config-reloader" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188860 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="config-reloader" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188865 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="prom-label-proxy" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188871 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="prom-label-proxy" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188877 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-web" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188882 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-web" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188925 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-metric" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188935 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="519aef38-8c7a-4de4-8224-6274fb384ddf" containerName="registry" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188941 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="config-reloader" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188946 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="prom-label-proxy" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188951 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="alertmanager" Jul 17 19:58:54.188954 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188956 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy" Jul 17 19:58:54.189386 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.188963 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" containerName="kube-rbac-proxy-web" Jul 17 19:58:54.189883 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.189852 2578 scope.go:117] "RemoveContainer" containerID="a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" Jul 17 19:58:54.190212 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.190183 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a\": container with ID starting with a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a not found: ID does not exist" containerID="a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" Jul 17 19:58:54.190283 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190211 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a"} err="failed to get container status \"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a\": rpc error: code = NotFound desc = could not find container \"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a\": container with ID starting with a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a not found: ID does not exist" Jul 17 19:58:54.190283 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190233 2578 scope.go:117] "RemoveContainer" containerID="dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" Jul 17 19:58:54.190507 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.190490 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea\": container with ID starting with dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea not found: ID does not exist" containerID="dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" Jul 17 19:58:54.190563 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190515 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea"} err="failed to get container status \"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea\": rpc error: code = NotFound desc = could not find container \"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea\": container with ID starting with dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea not found: ID does not exist" Jul 17 19:58:54.190563 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190531 2578 scope.go:117] "RemoveContainer" containerID="460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" Jul 17 19:58:54.190790 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.190773 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b\": container with ID starting with 460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b not found: ID does not exist" containerID="460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" Jul 17 19:58:54.190845 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190793 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b"} err="failed to get container status \"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b\": rpc error: code = NotFound desc = could not find container \"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b\": container with ID starting with 460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b not found: ID does not exist" Jul 17 19:58:54.190845 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.190807 2578 scope.go:117] "RemoveContainer" containerID="ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" Jul 17 19:58:54.191008 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.190993 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246\": container with ID starting with ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246 not found: ID does not exist" containerID="ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" Jul 17 19:58:54.191046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191012 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246"} err="failed to get container status \"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246\": rpc error: code = NotFound desc = could not find container \"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246\": container with ID starting with ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246 not found: ID does not exist" Jul 17 19:58:54.191046 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191024 2578 scope.go:117] "RemoveContainer" containerID="5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" Jul 17 19:58:54.191225 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.191206 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5\": container with ID starting with 5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5 not found: ID does not exist" containerID="5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" Jul 17 19:58:54.191266 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191231 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5"} err="failed to get container status \"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5\": rpc error: code = NotFound desc = could not find container \"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5\": container with ID starting with 5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5 not found: ID does not exist" Jul 17 19:58:54.191266 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191248 2578 scope.go:117] "RemoveContainer" containerID="38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" Jul 17 19:58:54.191516 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.191498 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf\": container with ID starting with 38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf not found: ID does not exist" containerID="38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" Jul 17 19:58:54.191571 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191522 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf"} err="failed to get container status \"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf\": rpc error: code = NotFound desc = could not find container \"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf\": container with ID starting with 38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf not found: ID does not exist" Jul 17 19:58:54.191571 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191537 2578 scope.go:117] "RemoveContainer" containerID="7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777" Jul 17 19:58:54.191764 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:58:54.191748 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777\": container with ID starting with 7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777 not found: ID does not exist" containerID="7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777" Jul 17 19:58:54.191808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191767 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777"} err="failed to get container status \"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777\": rpc error: code = NotFound desc = could not find container \"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777\": container with ID starting with 7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777 not found: ID does not exist" Jul 17 19:58:54.191808 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191778 2578 scope.go:117] "RemoveContainer" containerID="a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a" Jul 17 19:58:54.191965 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191948 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a"} err="failed to get container status \"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a\": rpc error: code = NotFound desc = could not find container \"a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a\": container with ID starting with a7f434c8a4278bd2a2964e042c1f15e06926e9f21f5d85aadd093fde2d33d44a not found: ID does not exist" Jul 17 19:58:54.192003 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.191966 2578 scope.go:117] "RemoveContainer" containerID="dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea" Jul 17 19:58:54.192147 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192132 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea"} err="failed to get container status \"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea\": rpc error: code = NotFound desc = could not find container \"dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea\": container with ID starting with dfb596ebac24188317e507f2144d260abb580bbeaacede2e92261bff0e0f17ea not found: ID does not exist" Jul 17 19:58:54.192147 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192145 2578 scope.go:117] "RemoveContainer" containerID="460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b" Jul 17 19:58:54.192335 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192312 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b"} err="failed to get container status \"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b\": rpc error: code = NotFound desc = could not find container \"460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b\": container with ID starting with 460139b6ec94a419287d32bad47de35797ccfb4ae3cbbfd3931f3af92906156b not found: ID does not exist" Jul 17 19:58:54.192377 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192335 2578 scope.go:117] "RemoveContainer" containerID="ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246" Jul 17 19:58:54.192548 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192530 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246"} err="failed to get container status \"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246\": rpc error: code = NotFound desc = could not find container \"ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246\": container with ID starting with ca416609d035fdce255cde9ad1b9492883f4edf29db8ed9ce88be5c6f6854246 not found: ID does not exist" Jul 17 19:58:54.192598 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192550 2578 scope.go:117] "RemoveContainer" containerID="5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5" Jul 17 19:58:54.192779 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192762 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5"} err="failed to get container status \"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5\": rpc error: code = NotFound desc = could not find container \"5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5\": container with ID starting with 5977ce281c1cbddb4ab38f1c0a833f9257c195662a347aa3d1e5a226efa640d5 not found: ID does not exist" Jul 17 19:58:54.192827 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192780 2578 scope.go:117] "RemoveContainer" containerID="38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf" Jul 17 19:58:54.193006 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.192970 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf"} err="failed to get container status \"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf\": rpc error: code = NotFound desc = could not find container \"38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf\": container with ID starting with 38b1ce2f7c924a3019c403f78608209881a836eab41690ed83a7818b0ce37fdf not found: ID does not exist" Jul 17 19:58:54.193049 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.193006 2578 scope.go:117] "RemoveContainer" containerID="7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777" Jul 17 19:58:54.193189 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.193173 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777"} err="failed to get container status \"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777\": rpc error: code = NotFound desc = could not find container \"7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777\": container with ID starting with 7fefe089ef1f4218733f9c65fd2356194d4fc505cc0161360889ee5dc09b2777 not found: ID does not exist" Jul 17 19:58:54.194077 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.194060 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.196173 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 19:58:54.196292 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196146 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 19:58:54.196292 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 19:58:54.196519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196498 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 19:58:54.196688 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196565 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 19:58:54.196688 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196596 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 19:58:54.196688 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196649 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d4xzg\"" Jul 17 19:58:54.196864 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196649 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 19:58:54.196864 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.196601 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 19:58:54.201781 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.201763 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 19:58:54.206113 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.206088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:54.386649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-web-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xxcb\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-kube-api-access-2xxcb\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-tls-assets\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-config-out\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-config-volume\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.386886 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.387135 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.387135 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.386927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488260 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488403 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488575 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488663 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488719 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-web-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488764 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xxcb\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-kube-api-access-2xxcb\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488815 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-tls-assets\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488815 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-config-out\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488920 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-config-volume\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.488920 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.488865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.489656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.489335 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.489656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.489444 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.489656 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.489489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763ce8d8-936a-4a7d-8d43-d61373c27445-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.491423 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.491393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.491535 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.491512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.491599 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.491541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.491683 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.491602 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.491683 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.491635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.492029 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.492011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-web-config\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.492101 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.492048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/763ce8d8-936a-4a7d-8d43-d61373c27445-config-volume\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.492270 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.492254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/763ce8d8-936a-4a7d-8d43-d61373c27445-config-out\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.493253 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.493237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-tls-assets\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.496415 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.496399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xxcb\" (UniqueName: \"kubernetes.io/projected/763ce8d8-936a-4a7d-8d43-d61373c27445-kube-api-access-2xxcb\") pod \"alertmanager-main-0\" (UID: \"763ce8d8-936a-4a7d-8d43-d61373c27445\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.503320 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.503299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 19:58:54.638126 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:54.637023 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 19:58:54.642169 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:58:54.642135 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763ce8d8_936a_4a7d_8d43_d61373c27445.slice/crio-61fcf98841ac8985aff974cdcadbb92d5ede9e989a96b146da07f863398a9ff1 WatchSource:0}: Error finding container 61fcf98841ac8985aff974cdcadbb92d5ede9e989a96b146da07f863398a9ff1: Status 404 returned error can't find the container with id 61fcf98841ac8985aff974cdcadbb92d5ede9e989a96b146da07f863398a9ff1 Jul 17 19:58:55.140213 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:55.140182 2578 generic.go:356] "Generic (PLEG): container finished" podID="763ce8d8-936a-4a7d-8d43-d61373c27445" containerID="b1727853173ac13e0ee2c08a27ec72da76b753acaa65cebb81a08f60c4c5f884" exitCode=0 Jul 17 19:58:55.140596 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:55.140227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerDied","Data":"b1727853173ac13e0ee2c08a27ec72da76b753acaa65cebb81a08f60c4c5f884"} Jul 17 19:58:55.140596 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:55.140275 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"61fcf98841ac8985aff974cdcadbb92d5ede9e989a96b146da07f863398a9ff1"} Jul 17 19:58:55.253104 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:55.253078 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd73de37-1ba8-4110-a18f-f36a05d53a7e" path="/var/lib/kubelet/pods/bd73de37-1ba8-4110-a18f-f36a05d53a7e/volumes" Jul 17 19:58:56.147145 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147113 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"a37606d450c9bcd7dcf4d885dfc75ffe14329074ed983793bcb1a49280baca8c"} Jul 17 19:58:56.147494 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147201 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"722de3e0c5db8b8e2540efc7ccd9dc2b7d0ac96c606073fc0c971793ff7de58c"} Jul 17 19:58:56.147494 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"0497e19a091c5bb6997f8024679d02509155eefffa22ee012ca021812e86d435"} Jul 17 19:58:56.147494 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147255 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"a396c724acd44f7be16d4fa64929ed8434b6c2078c19efccc1b0b2f4384359ce"} Jul 17 19:58:56.147494 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"f65f67066dc44798a51c788217ff10e17872ce5eda208a8667f51b0d4ceee533"} Jul 17 19:58:56.147494 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.147298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"763ce8d8-936a-4a7d-8d43-d61373c27445","Type":"ContainerStarted","Data":"efb21da020170086adc59592d68fc5ce35072d75a393bf8fc22f29338afed542"} Jul 17 19:58:56.171380 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.170927 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.155278876 podStartE2EDuration="2.170910884s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.632008ms" isStatefulPod=false podCreationTimestamp="2026-07-17 19:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:58:56.169784276 +0000 UTC m=+171.425015693" watchObservedRunningTime="2026-07-17 19:58:56.170910884 +0000 UTC m=+171.426142303" Jul 17 19:58:56.599862 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.599777 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-776b8bfc58-r8wqr"] Jul 17 19:58:56.603464 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.603438 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604114 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604259 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-metrics-client-ca\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604259 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgfkg\" (UniqueName: \"kubernetes.io/projected/fe3b9b08-7538-4749-83d7-dd10c7c1295c-kube-api-access-hgfkg\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604259 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604259 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-serving-certs-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604466 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-federate-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604466 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.604466 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.604387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.605532 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.605513 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zftwp\"" Jul 17 19:58:56.605532 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.605524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 17 19:58:56.605840 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.605821 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 17 19:58:56.605884 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.605860 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 17 19:58:56.605983 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.605965 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 17 19:58:56.606203 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.606189 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 17 19:58:56.610771 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.610751 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 17 19:58:56.614413 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.614392 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-776b8bfc58-r8wqr"] Jul 17 19:58:56.705023 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.704985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-metrics-client-ca\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgfkg\" (UniqueName: \"kubernetes.io/projected/fe3b9b08-7538-4749-83d7-dd10c7c1295c-kube-api-access-hgfkg\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-serving-certs-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-federate-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705214 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.705482 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.705311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.706159 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.706134 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-serving-certs-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.706280 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.706178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-metrics-client-ca\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.706280 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.706218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.707815 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.707789 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.707922 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.707829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-telemeter-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.708069 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.708050 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.708124 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.708053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe3b9b08-7538-4749-83d7-dd10c7c1295c-federate-client-tls\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.718070 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.718048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgfkg\" (UniqueName: \"kubernetes.io/projected/fe3b9b08-7538-4749-83d7-dd10c7c1295c-kube-api-access-hgfkg\") pod \"telemeter-client-776b8bfc58-r8wqr\" (UID: \"fe3b9b08-7538-4749-83d7-dd10c7c1295c\") " pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:56.915993 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:56.915948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" Jul 17 19:58:57.050543 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:57.050520 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-776b8bfc58-r8wqr"] Jul 17 19:58:57.052555 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:58:57.052535 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3b9b08_7538_4749_83d7_dd10c7c1295c.slice/crio-53944d9af155533174d3ef3d346e045f3dec2c840e3f903d897e3baab0ebf1b8 WatchSource:0}: Error finding container 53944d9af155533174d3ef3d346e045f3dec2c840e3f903d897e3baab0ebf1b8: Status 404 returned error can't find the container with id 53944d9af155533174d3ef3d346e045f3dec2c840e3f903d897e3baab0ebf1b8 Jul 17 19:58:57.152777 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:58:57.152741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" event={"ID":"fe3b9b08-7538-4749-83d7-dd10c7c1295c","Type":"ContainerStarted","Data":"53944d9af155533174d3ef3d346e045f3dec2c840e3f903d897e3baab0ebf1b8"} Jul 17 19:59:00.169639 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.169593 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" event={"ID":"fe3b9b08-7538-4749-83d7-dd10c7c1295c","Type":"ContainerStarted","Data":"18cf66c69a1012a8225c8b201368cf801d84379abd64ffbf93f4ae89b4ed9da8"} Jul 17 19:59:00.169996 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.169660 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" event={"ID":"fe3b9b08-7538-4749-83d7-dd10c7c1295c","Type":"ContainerStarted","Data":"50de54651e4fa56f5a36c3ca10bd0338826a8a8e16775917eb76d9a6e0cda3a8"} Jul 17 19:59:00.169996 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.169681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" event={"ID":"fe3b9b08-7538-4749-83d7-dd10c7c1295c","Type":"ContainerStarted","Data":"67f0cab89389f6a52fecdaaaf95b311669ef008dad6e76b37406ba7ed56a6e02"} Jul 17 19:59:00.901519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.901463 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-776b8bfc58-r8wqr" podStartSLOduration=2.818559029 podStartE2EDuration="4.901441503s" totalImagesPullingTime="2.082882474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:59:00.189665502 +0000 UTC m=+175.444896919" watchObservedRunningTime="2026-07-17 19:59:00.901441503 +0000 UTC m=+176.156672920" Jul 17 19:59:00.902388 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.902370 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 19:59:00.906071 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.906050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.917219 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.917193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 19:59:00.942286 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942255 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942286 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942519 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2gnx\" (UniqueName: \"kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942675 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:00.942675 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:00.942634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043599 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043599 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2gnx\" (UniqueName: \"kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.043848 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.043757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.044428 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.044404 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.044564 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.044431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.044564 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.044542 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.044693 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.044566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.046577 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.046551 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.046696 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.046679 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.051349 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.051331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2gnx\" (UniqueName: \"kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx\") pod \"console-bbd9d4d5b-tv7gw\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.217547 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.217439 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:01.357287 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:01.357252 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 19:59:01.360024 ip-10-0-142-80 kubenswrapper[2578]: W0717 19:59:01.359992 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8aff195_f04c_4278_917e_d3c6183adb95.slice/crio-3d9e586f78b2f06a958ee06e3ce82e861aa86a18071088f6fcea66d5aea48447 WatchSource:0}: Error finding container 3d9e586f78b2f06a958ee06e3ce82e861aa86a18071088f6fcea66d5aea48447: Status 404 returned error can't find the container with id 3d9e586f78b2f06a958ee06e3ce82e861aa86a18071088f6fcea66d5aea48447 Jul 17 19:59:02.179091 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:02.179056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbd9d4d5b-tv7gw" event={"ID":"a8aff195-f04c-4278-917e-d3c6183adb95","Type":"ContainerStarted","Data":"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764"} Jul 17 19:59:02.179271 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:02.179110 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbd9d4d5b-tv7gw" event={"ID":"a8aff195-f04c-4278-917e-d3c6183adb95","Type":"ContainerStarted","Data":"3d9e586f78b2f06a958ee06e3ce82e861aa86a18071088f6fcea66d5aea48447"} Jul 17 19:59:02.195844 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:02.195795 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbd9d4d5b-tv7gw" podStartSLOduration=2.195777231 podStartE2EDuration="2.195777231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:59:02.194229214 +0000 UTC m=+177.449460631" watchObservedRunningTime="2026-07-17 19:59:02.195777231 +0000 UTC m=+177.451008647" Jul 17 19:59:11.217800 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:11.217767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:11.218192 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:11.217822 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:11.223576 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:11.223549 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:12.225158 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:12.225127 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 19:59:12.269635 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:12.269586 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:59:37.298411 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.298370 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dcf74c77d-tbkmf" podUID="705d8a7d-1f39-4efa-81c9-954861267c4d" containerName="console" containerID="cri-o://609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b" gracePeriod=15 Jul 17 19:59:37.548252 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.548230 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:59:37.663225 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663194 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663243 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9sjw\" (UniqueName: \"kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663289 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663322 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663339 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663383 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663649 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663385 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle\") pod \"705d8a7d-1f39-4efa-81c9-954861267c4d\" (UID: \"705d8a7d-1f39-4efa-81c9-954861267c4d\") " Jul 17 19:59:37.663710 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663675 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:59:37.663873 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663840 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config" (OuterVolumeSpecName: "console-config") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:59:37.663967 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.663866 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca" (OuterVolumeSpecName: "service-ca") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:59:37.664092 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.664071 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:59:37.665642 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.665598 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:59:37.665755 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.665663 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:59:37.665755 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.665741 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw" (OuterVolumeSpecName: "kube-api-access-f9sjw") pod "705d8a7d-1f39-4efa-81c9-954861267c4d" (UID: "705d8a7d-1f39-4efa-81c9-954861267c4d"). InnerVolumeSpecName "kube-api-access-f9sjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:59:37.764238 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764199 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-oauth-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764251 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9sjw\" (UniqueName: \"kubernetes.io/projected/705d8a7d-1f39-4efa-81c9-954861267c4d-kube-api-access-f9sjw\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764263 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-oauth-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764271 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705d8a7d-1f39-4efa-81c9-954861267c4d-console-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764280 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-console-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764289 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:37.764399 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:37.764298 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d8a7d-1f39-4efa-81c9-954861267c4d-trusted-ca-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 19:59:38.330654 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.330604 2578 generic.go:356] "Generic (PLEG): container finished" podID="705d8a7d-1f39-4efa-81c9-954861267c4d" containerID="609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b" exitCode=0 Jul 17 19:59:38.331062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.330669 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dcf74c77d-tbkmf" Jul 17 19:59:38.331062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.330710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dcf74c77d-tbkmf" event={"ID":"705d8a7d-1f39-4efa-81c9-954861267c4d","Type":"ContainerDied","Data":"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b"} Jul 17 19:59:38.331062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.330760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dcf74c77d-tbkmf" event={"ID":"705d8a7d-1f39-4efa-81c9-954861267c4d","Type":"ContainerDied","Data":"245ea2635ec0b8ddea4b1b85137caf678500cf3965c80ed3946d4b98490038ec"} Jul 17 19:59:38.331062 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.330781 2578 scope.go:117] "RemoveContainer" containerID="609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b" Jul 17 19:59:38.339638 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.339605 2578 scope.go:117] "RemoveContainer" containerID="609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b" Jul 17 19:59:38.339889 ip-10-0-142-80 kubenswrapper[2578]: E0717 19:59:38.339871 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b\": container with ID starting with 609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b not found: ID does not exist" containerID="609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b" Jul 17 19:59:38.339935 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.339899 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b"} err="failed to get container status \"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b\": rpc error: code = NotFound desc = could not find container \"609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b\": container with ID starting with 609aebd832dea150b1930114ca1c486ac98e86f3cbc3b3b63678de7f3c9fac8b not found: ID does not exist" Jul 17 19:59:38.351376 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.351347 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:59:38.354312 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:38.354292 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dcf74c77d-tbkmf"] Jul 17 19:59:39.258333 ip-10-0-142-80 kubenswrapper[2578]: I0717 19:59:39.258298 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="705d8a7d-1f39-4efa-81c9-954861267c4d" path="/var/lib/kubelet/pods/705d8a7d-1f39-4efa-81c9-954861267c4d/volumes" Jul 17 20:00:26.143606 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.143570 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:00:26.144728 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.144704 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="705d8a7d-1f39-4efa-81c9-954861267c4d" containerName="console" Jul 17 20:00:26.144873 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.144862 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="705d8a7d-1f39-4efa-81c9-954861267c4d" containerName="console" Jul 17 20:00:26.145074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.145062 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="705d8a7d-1f39-4efa-81c9-954861267c4d" containerName="console" Jul 17 20:00:26.148279 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.148257 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.157663 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.157634 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:00:26.281542 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281542 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzv8\" (UniqueName: \"kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.281833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.281807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzv8\" (UniqueName: \"kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.382980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.382875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.383564 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.383538 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.383699 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.383589 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.383699 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.383591 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.383873 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.383855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.385316 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.385294 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.385392 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.385377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.390256 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.390215 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzv8\" (UniqueName: \"kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8\") pod \"console-6fc4db4f7b-w4l9f\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.465840 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.465737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:26.600752 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:26.600728 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:00:26.603066 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:00:26.603043 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03312d05_75a2_4a8a_ab50_db0e7dc6d2d9.slice/crio-a9d66d80234936bd45ac3de9cc193ff287d903b1efeb62c42c9096943e03a31f WatchSource:0}: Error finding container a9d66d80234936bd45ac3de9cc193ff287d903b1efeb62c42c9096943e03a31f: Status 404 returned error can't find the container with id a9d66d80234936bd45ac3de9cc193ff287d903b1efeb62c42c9096943e03a31f Jul 17 20:00:27.529198 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:27.529166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc4db4f7b-w4l9f" event={"ID":"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9","Type":"ContainerStarted","Data":"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae"} Jul 17 20:00:27.529580 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:27.529218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc4db4f7b-w4l9f" event={"ID":"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9","Type":"ContainerStarted","Data":"a9d66d80234936bd45ac3de9cc193ff287d903b1efeb62c42c9096943e03a31f"} Jul 17 20:00:27.545002 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:27.544947 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc4db4f7b-w4l9f" podStartSLOduration=1.544928793 podStartE2EDuration="1.544928793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:00:27.544141664 +0000 UTC m=+262.799373081" watchObservedRunningTime="2026-07-17 20:00:27.544928793 +0000 UTC m=+262.800160211" Jul 17 20:00:36.466856 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:36.466804 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:36.467405 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:36.466880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:36.471734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:36.471715 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:36.568312 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:36.568286 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:00:36.617425 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:36.617386 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 20:00:55.542753 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.542694 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm"] Jul 17 20:00:55.546497 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.546478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.549022 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.549002 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 20:00:55.549389 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.549374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5hqv6\"" Jul 17 20:00:55.549845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.549831 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 20:00:55.556884 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.556863 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm"] Jul 17 20:00:55.624955 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.624916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.625131 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.624963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nbq6\" (UniqueName: \"kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.625131 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.625020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.725377 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.725342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.725377 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.725385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nbq6\" (UniqueName: \"kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.725592 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.725424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.725759 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.725739 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.725798 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.725772 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.739168 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.739136 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nbq6\" (UniqueName: \"kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.856396 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.856299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:00:55.992275 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:55.992254 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm"] Jul 17 20:00:55.994421 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:00:55.994391 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c528377_1c0a_42ba_a2ac_f724fec88c72.slice/crio-183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39 WatchSource:0}: Error finding container 183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39: Status 404 returned error can't find the container with id 183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39 Jul 17 20:00:56.645879 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:00:56.645841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" event={"ID":"2c528377-1c0a-42ba-a2ac-f724fec88c72","Type":"ContainerStarted","Data":"183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39"} Jul 17 20:01:01.647045 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:01.646999 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bbd9d4d5b-tv7gw" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" containerName="console" containerID="cri-o://740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764" gracePeriod=15 Jul 17 20:01:02.221729 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.221687 2578 patch_prober.go:28] interesting pod/console-bbd9d4d5b-tv7gw container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body= Jul 17 20:01:02.221925 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.221749 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-bbd9d4d5b-tv7gw" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" Jul 17 20:01:02.867081 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.867055 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 20:01:02.989996 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.989913 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.989996 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.989950 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.989996 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.989972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.990253 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990028 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.990253 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990045 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.990253 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990063 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2gnx\" (UniqueName: \"kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.990253 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990139 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config\") pod \"a8aff195-f04c-4278-917e-d3c6183adb95\" (UID: \"a8aff195-f04c-4278-917e-d3c6183adb95\") " Jul 17 20:01:02.990452 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990388 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:01:02.990644 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990595 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:01:02.990644 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990636 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:01:02.990757 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.990687 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config" (OuterVolumeSpecName: "console-config") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:01:02.992136 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.992111 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:01:02.992299 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.992274 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx" (OuterVolumeSpecName: "kube-api-access-m2gnx") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "kube-api-access-m2gnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:01:02.992358 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:02.992279 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8aff195-f04c-4278-917e-d3c6183adb95" (UID: "a8aff195-f04c-4278-917e-d3c6183adb95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:01:03.091074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091021 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-console-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091066 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-oauth-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091076 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-oauth-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091089 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8aff195-f04c-4278-917e-d3c6183adb95-console-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091097 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-trusted-ca-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091368 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091106 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8aff195-f04c-4278-917e-d3c6183adb95-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.091368 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.091114 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2gnx\" (UniqueName: \"kubernetes.io/projected/a8aff195-f04c-4278-917e-d3c6183adb95-kube-api-access-m2gnx\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:03.676333 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.676291 2578 generic.go:356] "Generic (PLEG): container finished" podID="a8aff195-f04c-4278-917e-d3c6183adb95" containerID="740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764" exitCode=0 Jul 17 20:01:03.676530 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.676365 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbd9d4d5b-tv7gw" Jul 17 20:01:03.676530 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.676382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbd9d4d5b-tv7gw" event={"ID":"a8aff195-f04c-4278-917e-d3c6183adb95","Type":"ContainerDied","Data":"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764"} Jul 17 20:01:03.676530 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.676482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbd9d4d5b-tv7gw" event={"ID":"a8aff195-f04c-4278-917e-d3c6183adb95","Type":"ContainerDied","Data":"3d9e586f78b2f06a958ee06e3ce82e861aa86a18071088f6fcea66d5aea48447"} Jul 17 20:01:03.676530 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.676501 2578 scope.go:117] "RemoveContainer" containerID="740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764" Jul 17 20:01:03.678190 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.678165 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerID="831e52c4c4dfe528487b9ec236e3eab695eb6e897573cd223060cc7ad906d3fe" exitCode=0 Jul 17 20:01:03.678383 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.678245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" event={"ID":"2c528377-1c0a-42ba-a2ac-f724fec88c72","Type":"ContainerDied","Data":"831e52c4c4dfe528487b9ec236e3eab695eb6e897573cd223060cc7ad906d3fe"} Jul 17 20:01:03.685271 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.685254 2578 scope.go:117] "RemoveContainer" containerID="740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764" Jul 17 20:01:03.685516 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:01:03.685497 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764\": container with ID starting with 740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764 not found: ID does not exist" containerID="740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764" Jul 17 20:01:03.685572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.685524 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764"} err="failed to get container status \"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764\": rpc error: code = NotFound desc = could not find container \"740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764\": container with ID starting with 740c8259a4f54bc2ffb7fa5d483b2151ca3d5acc6c92a139838789d9127b2764 not found: ID does not exist" Jul 17 20:01:03.690597 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.690575 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 20:01:03.694193 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:03.694173 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bbd9d4d5b-tv7gw"] Jul 17 20:01:05.133982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:05.133951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:01:05.136305 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:05.136229 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:01:05.151316 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:05.151291 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 20:01:05.254678 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:05.254524 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" path="/var/lib/kubelet/pods/a8aff195-f04c-4278-917e-d3c6183adb95/volumes" Jul 17 20:01:06.694805 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:06.694763 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerID="74a2b3cd2af3c81ae255ad3d981883757af30f29ba365df618ac885cf8937514" exitCode=0 Jul 17 20:01:06.695302 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:06.694846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" event={"ID":"2c528377-1c0a-42ba-a2ac-f724fec88c72","Type":"ContainerDied","Data":"74a2b3cd2af3c81ae255ad3d981883757af30f29ba365df618ac885cf8937514"} Jul 17 20:01:06.696102 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:06.696079 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 20:01:14.733198 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:14.733166 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerID="d37e3a71e3362d7653fa8cae1ff6c60e1f6c0106109134332fb794c0ab9a3179" exitCode=0 Jul 17 20:01:14.733672 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:14.733244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" event={"ID":"2c528377-1c0a-42ba-a2ac-f724fec88c72","Type":"ContainerDied","Data":"d37e3a71e3362d7653fa8cae1ff6c60e1f6c0106109134332fb794c0ab9a3179"} Jul 17 20:01:15.862698 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:15.862674 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:01:16.009017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.008923 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util\") pod \"2c528377-1c0a-42ba-a2ac-f724fec88c72\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " Jul 17 20:01:16.009017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.008978 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle\") pod \"2c528377-1c0a-42ba-a2ac-f724fec88c72\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " Jul 17 20:01:16.009017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.009010 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nbq6\" (UniqueName: \"kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6\") pod \"2c528377-1c0a-42ba-a2ac-f724fec88c72\" (UID: \"2c528377-1c0a-42ba-a2ac-f724fec88c72\") " Jul 17 20:01:16.009647 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.009594 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle" (OuterVolumeSpecName: "bundle") pod "2c528377-1c0a-42ba-a2ac-f724fec88c72" (UID: "2c528377-1c0a-42ba-a2ac-f724fec88c72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:16.011248 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.011202 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6" (OuterVolumeSpecName: "kube-api-access-8nbq6") pod "2c528377-1c0a-42ba-a2ac-f724fec88c72" (UID: "2c528377-1c0a-42ba-a2ac-f724fec88c72"). InnerVolumeSpecName "kube-api-access-8nbq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:01:16.015091 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.015064 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util" (OuterVolumeSpecName: "util") pod "2c528377-1c0a-42ba-a2ac-f724fec88c72" (UID: "2c528377-1c0a-42ba-a2ac-f724fec88c72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:16.109950 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.109910 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:16.109950 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.109941 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c528377-1c0a-42ba-a2ac-f724fec88c72-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:16.109950 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.109954 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nbq6\" (UniqueName: \"kubernetes.io/projected/2c528377-1c0a-42ba-a2ac-f724fec88c72-kube-api-access-8nbq6\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:16.742709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.742685 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" Jul 17 20:01:16.742876 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.742679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540fdzbm" event={"ID":"2c528377-1c0a-42ba-a2ac-f724fec88c72","Type":"ContainerDied","Data":"183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39"} Jul 17 20:01:16.742876 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:16.742803 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="183fd2c4798f378d958844d07c4d50457e6946c40d3bcd13fe6442454b21ea39" Jul 17 20:01:23.438803 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.438771 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z"] Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439096 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="pull" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439106 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="pull" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439120 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="util" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439125 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="util" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439131 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" containerName="console" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439139 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" containerName="console" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439154 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="extract" Jul 17 20:01:23.439179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439159 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="extract" Jul 17 20:01:23.439425 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439207 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c528377-1c0a-42ba-a2ac-f724fec88c72" containerName="extract" Jul 17 20:01:23.439425 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.439216 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8aff195-f04c-4278-917e-d3c6183adb95" containerName="console" Jul 17 20:01:23.443112 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.443094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.446914 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.446893 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 17 20:01:23.447030 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.446940 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 17 20:01:23.447030 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.447000 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-94jnt\"" Jul 17 20:01:23.456351 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.456326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z"] Jul 17 20:01:23.574009 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.573970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb5vg\" (UniqueName: \"kubernetes.io/projected/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-kube-api-access-jb5vg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.574009 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.574018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.674529 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.674498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.674734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.674586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb5vg\" (UniqueName: \"kubernetes.io/projected/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-kube-api-access-jb5vg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.674920 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.674897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.681925 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.681899 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb5vg\" (UniqueName: \"kubernetes.io/projected/23c99aa9-8bbc-4ee1-bb63-e1ef69654778-kube-api-access-jb5vg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mv8z\" (UID: \"23c99aa9-8bbc-4ee1-bb63-e1ef69654778\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.752659 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.752544 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" Jul 17 20:01:23.886527 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:23.886503 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z"] Jul 17 20:01:23.889465 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:01:23.889440 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c99aa9_8bbc_4ee1_bb63_e1ef69654778.slice/crio-5a9209606324ae02d942786708fa911daf55208b9a7d84b819c4f8ef2e5126f2 WatchSource:0}: Error finding container 5a9209606324ae02d942786708fa911daf55208b9a7d84b819c4f8ef2e5126f2: Status 404 returned error can't find the container with id 5a9209606324ae02d942786708fa911daf55208b9a7d84b819c4f8ef2e5126f2 Jul 17 20:01:24.777928 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:24.777886 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" event={"ID":"23c99aa9-8bbc-4ee1-bb63-e1ef69654778","Type":"ContainerStarted","Data":"5a9209606324ae02d942786708fa911daf55208b9a7d84b819c4f8ef2e5126f2"} Jul 17 20:01:25.785700 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:25.785663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" event={"ID":"23c99aa9-8bbc-4ee1-bb63-e1ef69654778","Type":"ContainerStarted","Data":"280fe7a05c7da15c452fc0d9be2205aba9eb23ff5a4e65ac85ccd90b4138b8d5"} Jul 17 20:01:25.809425 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:25.809370 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mv8z" podStartSLOduration=1.100635259 podStartE2EDuration="2.809352674s" totalImagesPullingTime="1.708717415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:01:25.808239178 +0000 UTC m=+321.063470595" watchObservedRunningTime="2026-07-17 20:01:25.809352674 +0000 UTC m=+321.064584091" Jul 17 20:01:27.177031 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.176999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf"] Jul 17 20:01:27.179309 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.179289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.181929 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.181907 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 20:01:27.182059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.181907 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 20:01:27.182374 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.182355 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5hqv6\"" Jul 17 20:01:27.190311 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.190287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf"] Jul 17 20:01:27.308037 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.307988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.308237 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.308073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6kn2\" (UniqueName: \"kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.308237 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.308100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.409058 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.409017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6kn2\" (UniqueName: \"kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.409058 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.409063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.409259 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.409104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.409556 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.409535 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.409589 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.409550 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.418324 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.418297 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6kn2\" (UniqueName: \"kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.488948 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.488866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:27.616001 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.615977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf"] Jul 17 20:01:27.618374 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:01:27.618347 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5f61e9_ce70_4b36_aa7e_752a01c0053b.slice/crio-ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c WatchSource:0}: Error finding container ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c: Status 404 returned error can't find the container with id ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c Jul 17 20:01:27.694479 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.694442 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-42762"] Jul 17 20:01:27.696821 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.696806 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.699030 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.699001 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-s6xk9\"" Jul 17 20:01:27.699030 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.699026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 17 20:01:27.699245 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.699052 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 17 20:01:27.705953 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.705925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-42762"] Jul 17 20:01:27.798128 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.798035 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerID="b77fcc64a04fe40b665509e9a420fb8c3174b0bd3c1f1706e1019cafd90ccb09" exitCode=0 Jul 17 20:01:27.798128 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.798119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" event={"ID":"fa5f61e9-ce70-4b36-aa7e-752a01c0053b","Type":"ContainerDied","Data":"b77fcc64a04fe40b665509e9a420fb8c3174b0bd3c1f1706e1019cafd90ccb09"} Jul 17 20:01:27.798347 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.798189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" event={"ID":"fa5f61e9-ce70-4b36-aa7e-752a01c0053b","Type":"ContainerStarted","Data":"ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c"} Jul 17 20:01:27.813045 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.813014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.813248 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.813080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6g6b\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-kube-api-access-p6g6b\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.913995 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.913959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6g6b\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-kube-api-access-p6g6b\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.914170 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.914024 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.921463 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.921432 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:27.921667 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:27.921648 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6g6b\" (UniqueName: \"kubernetes.io/projected/fd454a79-165a-40a9-913d-6cd7900e390b-kube-api-access-p6g6b\") pod \"cert-manager-webhook-6486ffd997-42762\" (UID: \"fd454a79-165a-40a9-913d-6cd7900e390b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:28.020734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:28.020706 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:28.148752 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:28.148728 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-42762"] Jul 17 20:01:28.151056 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:01:28.151029 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd454a79_165a_40a9_913d_6cd7900e390b.slice/crio-35aa6a16e7d2cbd831ac831fb6dd6570db9d8c38642ed4d558b8520b5210e08d WatchSource:0}: Error finding container 35aa6a16e7d2cbd831ac831fb6dd6570db9d8c38642ed4d558b8520b5210e08d: Status 404 returned error can't find the container with id 35aa6a16e7d2cbd831ac831fb6dd6570db9d8c38642ed4d558b8520b5210e08d Jul 17 20:01:28.804763 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:28.804679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" event={"ID":"fd454a79-165a-40a9-913d-6cd7900e390b","Type":"ContainerStarted","Data":"35aa6a16e7d2cbd831ac831fb6dd6570db9d8c38642ed4d558b8520b5210e08d"} Jul 17 20:01:29.689533 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.689493 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-psg97"] Jul 17 20:01:29.692995 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.692969 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.695094 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.695074 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-crt5w\"" Jul 17 20:01:29.701193 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.700972 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-psg97"] Jul 17 20:01:29.732742 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.732708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2sp\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-kube-api-access-dz2sp\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.732910 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.732755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.833829 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.833795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2sp\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-kube-api-access-dz2sp\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.834270 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.833857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.842014 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.841980 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:29.842176 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:29.842079 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2sp\" (UniqueName: \"kubernetes.io/projected/33791aeb-0312-42e0-91b9-feb0d2ce53fb-kube-api-access-dz2sp\") pod \"cert-manager-cainjector-559cdd8c87-psg97\" (UID: \"33791aeb-0312-42e0-91b9-feb0d2ce53fb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:30.004957 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:30.004861 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" Jul 17 20:01:31.754355 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.754327 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-psg97"] Jul 17 20:01:31.756863 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:01:31.756835 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33791aeb_0312_42e0_91b9_feb0d2ce53fb.slice/crio-23425f05817e0d1cff348173e29b43de04b011942960b74e0a5417f0e24c5224 WatchSource:0}: Error finding container 23425f05817e0d1cff348173e29b43de04b011942960b74e0a5417f0e24c5224: Status 404 returned error can't find the container with id 23425f05817e0d1cff348173e29b43de04b011942960b74e0a5417f0e24c5224 Jul 17 20:01:31.821718 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.821683 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerID="2b6699244bae52da098286bd9c3bbb994028d6346e9a36220725f6a191612ea3" exitCode=0 Jul 17 20:01:31.821885 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.821767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" event={"ID":"fa5f61e9-ce70-4b36-aa7e-752a01c0053b","Type":"ContainerDied","Data":"2b6699244bae52da098286bd9c3bbb994028d6346e9a36220725f6a191612ea3"} Jul 17 20:01:31.823050 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.823025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" event={"ID":"33791aeb-0312-42e0-91b9-feb0d2ce53fb","Type":"ContainerStarted","Data":"23425f05817e0d1cff348173e29b43de04b011942960b74e0a5417f0e24c5224"} Jul 17 20:01:31.824499 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.824478 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" event={"ID":"fd454a79-165a-40a9-913d-6cd7900e390b","Type":"ContainerStarted","Data":"0dd0720af80c85063c1018d2aac4bf865db127ddde227340ae471c79d1efdd59"} Jul 17 20:01:31.824668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.824603 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:31.850782 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:31.850735 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" podStartSLOduration=1.382919073 podStartE2EDuration="4.850719418s" totalImagesPullingTime="3.467800345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:01:31.84960137 +0000 UTC m=+327.104832798" watchObservedRunningTime="2026-07-17 20:01:31.850719418 +0000 UTC m=+327.105950834" Jul 17 20:01:32.831133 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:32.831103 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerID="2343fbf9878a83575c4c80c25f06f7fe5a55fc067b2d95c89ac6a47f5321833f" exitCode=0 Jul 17 20:01:32.831569 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:32.831189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" event={"ID":"fa5f61e9-ce70-4b36-aa7e-752a01c0053b","Type":"ContainerDied","Data":"2343fbf9878a83575c4c80c25f06f7fe5a55fc067b2d95c89ac6a47f5321833f"} Jul 17 20:01:32.832761 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:32.832737 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" event={"ID":"33791aeb-0312-42e0-91b9-feb0d2ce53fb","Type":"ContainerStarted","Data":"9ccbc0decf8db9d172358490b8e308cce106ea2eb207edfac4c532c2164e1118"} Jul 17 20:01:32.862426 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:32.862323 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-psg97" podStartSLOduration=3.862309052 podStartE2EDuration="3.862309052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:01:32.860424642 +0000 UTC m=+328.115656058" watchObservedRunningTime="2026-07-17 20:01:32.862309052 +0000 UTC m=+328.117540468" Jul 17 20:01:33.964702 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:33.964680 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:34.074370 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.074339 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util\") pod \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " Jul 17 20:01:34.074548 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.074395 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6kn2\" (UniqueName: \"kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2\") pod \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " Jul 17 20:01:34.074548 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.074488 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle\") pod \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\" (UID: \"fa5f61e9-ce70-4b36-aa7e-752a01c0053b\") " Jul 17 20:01:34.074840 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.074819 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle" (OuterVolumeSpecName: "bundle") pod "fa5f61e9-ce70-4b36-aa7e-752a01c0053b" (UID: "fa5f61e9-ce70-4b36-aa7e-752a01c0053b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:34.076469 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.076446 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2" (OuterVolumeSpecName: "kube-api-access-k6kn2") pod "fa5f61e9-ce70-4b36-aa7e-752a01c0053b" (UID: "fa5f61e9-ce70-4b36-aa7e-752a01c0053b"). InnerVolumeSpecName "kube-api-access-k6kn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:01:34.079388 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.079350 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util" (OuterVolumeSpecName: "util") pod "fa5f61e9-ce70-4b36-aa7e-752a01c0053b" (UID: "fa5f61e9-ce70-4b36-aa7e-752a01c0053b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:34.175227 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.175180 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:34.175227 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.175219 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:34.175227 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.175229 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6kn2\" (UniqueName: \"kubernetes.io/projected/fa5f61e9-ce70-4b36-aa7e-752a01c0053b-kube-api-access-k6kn2\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:34.846872 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.846844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" Jul 17 20:01:34.847080 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.846843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frr5sf" event={"ID":"fa5f61e9-ce70-4b36-aa7e-752a01c0053b","Type":"ContainerDied","Data":"ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c"} Jul 17 20:01:34.847080 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:34.846965 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ede643ddd1be40357d1b8ad698e062db9c596c2141f8673e644a79d939bd816c" Jul 17 20:01:37.835252 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:37.835224 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-42762" Jul 17 20:01:47.450579 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450542 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl"] Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450924 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="extract" Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450937 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="extract" Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450952 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="util" Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450958 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="util" Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450966 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="pull" Jul 17 20:01:47.451012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.450972 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="pull" Jul 17 20:01:47.451230 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.451024 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5f61e9-ce70-4b36-aa7e-752a01c0053b" containerName="extract" Jul 17 20:01:47.455798 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.455779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.457795 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.457777 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 20:01:47.457957 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.457942 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 20:01:47.458581 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.458562 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5hqv6\"" Jul 17 20:01:47.462029 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.462004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl"] Jul 17 20:01:47.599067 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.599032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.599240 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.599072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.599240 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.599153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7cxx\" (UniqueName: \"kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.700403 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.700353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.700403 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.700406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.700685 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.700459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7cxx\" (UniqueName: \"kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.700781 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.700764 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.700880 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.700858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.708038 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.708014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7cxx\" (UniqueName: \"kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.767328 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.767293 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:47.899369 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.899345 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl"] Jul 17 20:01:47.901472 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:01:47.901438 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd440d7d1_aec3_4353_ae35_debc13b2d968.slice/crio-b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def WatchSource:0}: Error finding container b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def: Status 404 returned error can't find the container with id b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def Jul 17 20:01:47.907760 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:47.907734 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" event={"ID":"d440d7d1-aec3-4353-ae35-debc13b2d968","Type":"ContainerStarted","Data":"b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def"} Jul 17 20:01:48.913523 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:48.913488 2578 generic.go:356] "Generic (PLEG): container finished" podID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerID="c84cfa4571ef20bb401c20e8df509d9b2a76fab756da7f478020b0d6b614e6d6" exitCode=0 Jul 17 20:01:48.913919 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:48.913580 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" event={"ID":"d440d7d1-aec3-4353-ae35-debc13b2d968","Type":"ContainerDied","Data":"c84cfa4571ef20bb401c20e8df509d9b2a76fab756da7f478020b0d6b614e6d6"} Jul 17 20:01:49.926017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:49.925980 2578 generic.go:356] "Generic (PLEG): container finished" podID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerID="8989e1b6047277ab230134a971736e50d8d1fde960bf15a19bc28892ba842340" exitCode=0 Jul 17 20:01:49.926459 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:49.926056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" event={"ID":"d440d7d1-aec3-4353-ae35-debc13b2d968","Type":"ContainerDied","Data":"8989e1b6047277ab230134a971736e50d8d1fde960bf15a19bc28892ba842340"} Jul 17 20:01:50.932743 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:50.932711 2578 generic.go:356] "Generic (PLEG): container finished" podID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerID="924e92579f32127fe554e5d0af8ade40d3e888660514d5b8cb7b34431c6cea6a" exitCode=0 Jul 17 20:01:50.933128 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:50.932767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" event={"ID":"d440d7d1-aec3-4353-ae35-debc13b2d968","Type":"ContainerDied","Data":"924e92579f32127fe554e5d0af8ade40d3e888660514d5b8cb7b34431c6cea6a"} Jul 17 20:01:52.066434 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.066408 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:01:52.139776 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.139741 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle\") pod \"d440d7d1-aec3-4353-ae35-debc13b2d968\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " Jul 17 20:01:52.139982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.139801 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7cxx\" (UniqueName: \"kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx\") pod \"d440d7d1-aec3-4353-ae35-debc13b2d968\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " Jul 17 20:01:52.139982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.139835 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util\") pod \"d440d7d1-aec3-4353-ae35-debc13b2d968\" (UID: \"d440d7d1-aec3-4353-ae35-debc13b2d968\") " Jul 17 20:01:52.140560 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.140531 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle" (OuterVolumeSpecName: "bundle") pod "d440d7d1-aec3-4353-ae35-debc13b2d968" (UID: "d440d7d1-aec3-4353-ae35-debc13b2d968"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:52.141924 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.141903 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx" (OuterVolumeSpecName: "kube-api-access-d7cxx") pod "d440d7d1-aec3-4353-ae35-debc13b2d968" (UID: "d440d7d1-aec3-4353-ae35-debc13b2d968"). InnerVolumeSpecName "kube-api-access-d7cxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:01:52.145723 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.145699 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util" (OuterVolumeSpecName: "util") pod "d440d7d1-aec3-4353-ae35-debc13b2d968" (UID: "d440d7d1-aec3-4353-ae35-debc13b2d968"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:01:52.240552 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.240463 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:52.240552 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.240492 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7cxx\" (UniqueName: \"kubernetes.io/projected/d440d7d1-aec3-4353-ae35-debc13b2d968-kube-api-access-d7cxx\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:52.240552 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.240505 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d440d7d1-aec3-4353-ae35-debc13b2d968-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:01:52.943776 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.943745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" event={"ID":"d440d7d1-aec3-4353-ae35-debc13b2d968","Type":"ContainerDied","Data":"b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def"} Jul 17 20:01:52.943944 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.943790 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2d21b721aee7984a5749292e55c409c2acef2c671498d9440168d0faaf71def" Jul 17 20:01:52.943944 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:01:52.943794 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmtcdl" Jul 17 20:02:04.512379 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512295 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd"] Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512646 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="extract" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512659 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="extract" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512669 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="util" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512676 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="util" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512693 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="pull" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512698 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="pull" Jul 17 20:02:04.512845 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.512749 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d440d7d1-aec3-4353-ae35-debc13b2d968" containerName="extract" Jul 17 20:02:04.515227 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.515207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.517775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.517605 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 20:02:04.517775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.517632 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 20:02:04.517775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.517688 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 20:02:04.517775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.517699 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rss4j\"" Jul 17 20:02:04.517775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.517719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 20:02:04.531014 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.530990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd"] Jul 17 20:02:04.649099 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.649053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-webhook-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.649274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.649123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-manifests\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.649274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.649149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-tmp\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.649274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.649167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqql\" (UniqueName: \"kubernetes.io/projected/a975e802-bcdd-479e-a286-4c47cef98646-kube-api-access-4fqql\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.649274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.649262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750073 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750279 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-webhook-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750279 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-manifests\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750279 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-tmp\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750279 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqql\" (UniqueName: \"kubernetes.io/projected/a975e802-bcdd-479e-a286-4c47cef98646-kube-api-access-4fqql\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.750653 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.750599 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-manifests\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.752445 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.752421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a975e802-bcdd-479e-a286-4c47cef98646-tmp\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.752681 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.752666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.752681 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.752672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a975e802-bcdd-479e-a286-4c47cef98646-webhook-cert\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.757800 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.757775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqql\" (UniqueName: \"kubernetes.io/projected/a975e802-bcdd-479e-a286-4c47cef98646-kube-api-access-4fqql\") pod \"opendatahub-operator-controller-manager-67bdc7ddc4-b97bd\" (UID: \"a975e802-bcdd-479e-a286-4c47cef98646\") " pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.825697 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.825573 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:04.959388 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:04.959365 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd"] Jul 17 20:02:04.961648 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:02:04.961622 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda975e802_bcdd_479e_a286_4c47cef98646.slice/crio-99bb2b8a4127a6a68af7982767cb452702facfda1c963b5e3d775449ca2a5797 WatchSource:0}: Error finding container 99bb2b8a4127a6a68af7982767cb452702facfda1c963b5e3d775449ca2a5797: Status 404 returned error can't find the container with id 99bb2b8a4127a6a68af7982767cb452702facfda1c963b5e3d775449ca2a5797 Jul 17 20:02:05.001800 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:05.001766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" event={"ID":"a975e802-bcdd-479e-a286-4c47cef98646","Type":"ContainerStarted","Data":"99bb2b8a4127a6a68af7982767cb452702facfda1c963b5e3d775449ca2a5797"} Jul 17 20:02:08.021179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:08.021147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" event={"ID":"a975e802-bcdd-479e-a286-4c47cef98646","Type":"ContainerStarted","Data":"a15bc128e73165e1110f136d94961c1f061b6a984bbf7b48cad669c8ea12980e"} Jul 17 20:02:10.031503 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:10.031470 2578 generic.go:356] "Generic (PLEG): container finished" podID="a975e802-bcdd-479e-a286-4c47cef98646" containerID="a15bc128e73165e1110f136d94961c1f061b6a984bbf7b48cad669c8ea12980e" exitCode=0 Jul 17 20:02:10.031913 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:10.031546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" event={"ID":"a975e802-bcdd-479e-a286-4c47cef98646","Type":"ContainerDied","Data":"a15bc128e73165e1110f136d94961c1f061b6a984bbf7b48cad669c8ea12980e"} Jul 17 20:02:11.038190 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:11.038162 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" event={"ID":"a975e802-bcdd-479e-a286-4c47cef98646","Type":"ContainerStarted","Data":"6c0335c82085cca3f01f99e8cb85c065c8d8b3d106135188ea1c58e2336d877e"} Jul 17 20:02:11.038671 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:11.038369 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:11.058030 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:11.057975 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" podStartSLOduration=2.666840829 podStartE2EDuration="7.057954435s" totalImagesPullingTime="3.371555131s" totalInitContainerRuntime="1.019558475s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:02:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:02:11.056084579 +0000 UTC m=+366.311315997" watchObservedRunningTime="2026-07-17 20:02:11.057954435 +0000 UTC m=+366.313185852" Jul 17 20:02:22.045750 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:22.045719 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67bdc7ddc4-b97bd" Jul 17 20:02:31.601352 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.601313 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn"] Jul 17 20:02:31.604877 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.604855 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.606905 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.606883 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 17 20:02:31.607015 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.606888 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 17 20:02:31.607150 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.607137 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 17 20:02:31.607743 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.607728 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 17 20:02:31.607800 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.607768 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 17 20:02:31.607872 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.607856 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dq5kt\"" Jul 17 20:02:31.613838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.613815 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn"] Jul 17 20:02:31.690967 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.690938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.691108 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.690972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-metrics-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.691108 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.691084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f93ca144-8090-4bfc-bb10-f31923b91078-manager-config\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.691214 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.691117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87m2\" (UniqueName: \"kubernetes.io/projected/f93ca144-8090-4bfc-bb10-f31923b91078-kube-api-access-n87m2\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.792302 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.792269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.792302 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.792307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-metrics-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.792509 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.792411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f93ca144-8090-4bfc-bb10-f31923b91078-manager-config\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.792509 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.792441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87m2\" (UniqueName: \"kubernetes.io/projected/f93ca144-8090-4bfc-bb10-f31923b91078-kube-api-access-n87m2\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.793094 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.793060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f93ca144-8090-4bfc-bb10-f31923b91078-manager-config\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.794852 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.794829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-metrics-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.794950 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.794870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93ca144-8090-4bfc-bb10-f31923b91078-cert\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.800071 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.800049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87m2\" (UniqueName: \"kubernetes.io/projected/f93ca144-8090-4bfc-bb10-f31923b91078-kube-api-access-n87m2\") pod \"lws-controller-manager-7c54c94474-b79zn\" (UID: \"f93ca144-8090-4bfc-bb10-f31923b91078\") " pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:31.915352 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:31.915320 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:32.046180 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:32.046157 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn"] Jul 17 20:02:32.047987 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:02:32.047960 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93ca144_8090_4bfc_bb10_f31923b91078.slice/crio-89530fdc57057cf6f47fcdf9b1d6c2f4acdc7d1c3a1a21a886856a0a2708e0d6 WatchSource:0}: Error finding container 89530fdc57057cf6f47fcdf9b1d6c2f4acdc7d1c3a1a21a886856a0a2708e0d6: Status 404 returned error can't find the container with id 89530fdc57057cf6f47fcdf9b1d6c2f4acdc7d1c3a1a21a886856a0a2708e0d6 Jul 17 20:02:32.145467 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:32.145435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" event={"ID":"f93ca144-8090-4bfc-bb10-f31923b91078","Type":"ContainerStarted","Data":"89530fdc57057cf6f47fcdf9b1d6c2f4acdc7d1c3a1a21a886856a0a2708e0d6"} Jul 17 20:02:34.157706 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.157674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" event={"ID":"f93ca144-8090-4bfc-bb10-f31923b91078","Type":"ContainerStarted","Data":"1a020e5211b4d3f67b834bc09521564760eed804b3569842b3506874fb9e1e7f"} Jul 17 20:02:34.158173 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.157804 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:02:34.176259 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.176212 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" podStartSLOduration=1.415763894 podStartE2EDuration="3.176195628s" totalImagesPullingTime="1.760431734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:02:34.173544561 +0000 UTC m=+389.428775978" watchObservedRunningTime="2026-07-17 20:02:34.176195628 +0000 UTC m=+389.431427111" Jul 17 20:02:34.493041 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.492957 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm"] Jul 17 20:02:34.496463 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.496441 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.498900 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.498878 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 20:02:34.499015 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.498930 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 20:02:34.499015 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.498991 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5wcdl\"" Jul 17 20:02:34.507252 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.507229 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm"] Jul 17 20:02:34.617724 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.617688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ccc331-03b5-437d-8bfc-ac910c5479ac-tmp\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.617724 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.617743 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85ccc331-03b5-437d-8bfc-ac910c5479ac-tls-certs\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.617986 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.617766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4wqd\" (UniqueName: \"kubernetes.io/projected/85ccc331-03b5-437d-8bfc-ac910c5479ac-kube-api-access-p4wqd\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.718685 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.718655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ccc331-03b5-437d-8bfc-ac910c5479ac-tmp\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.718874 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.718712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85ccc331-03b5-437d-8bfc-ac910c5479ac-tls-certs\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.718874 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.718752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4wqd\" (UniqueName: \"kubernetes.io/projected/85ccc331-03b5-437d-8bfc-ac910c5479ac-kube-api-access-p4wqd\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.720900 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.720877 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ccc331-03b5-437d-8bfc-ac910c5479ac-tmp\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.721107 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.721091 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85ccc331-03b5-437d-8bfc-ac910c5479ac-tls-certs\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.727418 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.727395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4wqd\" (UniqueName: \"kubernetes.io/projected/85ccc331-03b5-437d-8bfc-ac910c5479ac-kube-api-access-p4wqd\") pod \"kube-auth-proxy-fdbd865b7-hcxrm\" (UID: \"85ccc331-03b5-437d-8bfc-ac910c5479ac\") " pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.806657 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.806558 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" Jul 17 20:02:34.966895 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:34.966871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm"] Jul 17 20:02:34.969073 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:02:34.969048 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ccc331_03b5_437d_8bfc_ac910c5479ac.slice/crio-876eb2cafe1378023735d3c7a8784d59dffde4c1e9f065fd8a96def0790a218c WatchSource:0}: Error finding container 876eb2cafe1378023735d3c7a8784d59dffde4c1e9f065fd8a96def0790a218c: Status 404 returned error can't find the container with id 876eb2cafe1378023735d3c7a8784d59dffde4c1e9f065fd8a96def0790a218c Jul 17 20:02:35.165863 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:35.165820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" event={"ID":"85ccc331-03b5-437d-8bfc-ac910c5479ac","Type":"ContainerStarted","Data":"876eb2cafe1378023735d3c7a8784d59dffde4c1e9f065fd8a96def0790a218c"} Jul 17 20:02:38.185736 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:38.185701 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" event={"ID":"85ccc331-03b5-437d-8bfc-ac910c5479ac","Type":"ContainerStarted","Data":"46677582f91d26967f00562fe28caa62d74097e4d639961187d4c7ac520c330a"} Jul 17 20:02:38.206019 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:38.205955 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-fdbd865b7-hcxrm" podStartSLOduration=1.397871889 podStartE2EDuration="4.205933511s" totalImagesPullingTime="2.808061622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:02:38.204286994 +0000 UTC m=+393.459518412" watchObservedRunningTime="2026-07-17 20:02:38.205933511 +0000 UTC m=+393.461164929" Jul 17 20:02:45.168119 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:02:45.168088 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c54c94474-b79zn" Jul 17 20:03:40.823836 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.823764 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk"] Jul 17 20:03:40.828652 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.828606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:40.830981 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.830956 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 20:03:40.830981 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.830970 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 20:03:40.831767 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.831753 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hf6p\"" Jul 17 20:03:40.834793 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:40.834769 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk"] Jul 17 20:03:41.016272 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.016236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.016429 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.016333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4d6c\" (UniqueName: \"kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.016471 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.016432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.117384 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.117279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d6c\" (UniqueName: \"kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.117572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.117386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.117572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.117417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.117821 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.117800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.117880 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.117830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.125693 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.125670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4d6c\" (UniqueName: \"kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.139284 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.139255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:41.279410 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.279379 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk"] Jul 17 20:03:41.281020 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:03:41.280995 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6055ab65_537a_41a1_87eb_9d9bae4fe02b.slice/crio-257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497 WatchSource:0}: Error finding container 257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497: Status 404 returned error can't find the container with id 257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497 Jul 17 20:03:41.500062 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.500031 2578 generic.go:356] "Generic (PLEG): container finished" podID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerID="25b4e4c04c7f6507bc54a64ce0d64cbe94f77041f5fb5b3c01c8a42ff7dc7411" exitCode=0 Jul 17 20:03:41.500228 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.500078 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" event={"ID":"6055ab65-537a-41a1-87eb-9d9bae4fe02b","Type":"ContainerDied","Data":"25b4e4c04c7f6507bc54a64ce0d64cbe94f77041f5fb5b3c01c8a42ff7dc7411"} Jul 17 20:03:41.500228 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.500118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" event={"ID":"6055ab65-537a-41a1-87eb-9d9bae4fe02b","Type":"ContainerStarted","Data":"257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497"} Jul 17 20:03:41.624049 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.624015 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb"] Jul 17 20:03:41.627547 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.627529 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.634628 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.634578 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb"] Jul 17 20:03:41.722196 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.722164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfb8d\" (UniqueName: \"kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.722376 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.722212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.722376 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.722263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.822899 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.822866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.823057 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.822931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfb8d\" (UniqueName: \"kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.823057 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.822980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.823353 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.823331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.823353 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.823347 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.831154 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.831119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfb8d\" (UniqueName: \"kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:41.937777 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:41.937735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:42.073847 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.073818 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb"] Jul 17 20:03:42.075306 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:03:42.075283 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732066e2_6d11_4713_b407_9404558d7e9b.slice/crio-5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95 WatchSource:0}: Error finding container 5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95: Status 404 returned error can't find the container with id 5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95 Jul 17 20:03:42.421136 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.421099 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9"] Jul 17 20:03:42.424734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.424713 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.428635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.428577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.428635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.428632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.428807 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.428691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fh5z\" (UniqueName: \"kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.433124 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.433099 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9"] Jul 17 20:03:42.507362 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.507276 2578 generic.go:356] "Generic (PLEG): container finished" podID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerID="059533ec3f3ed785453bd85a511934dd2fdffb40b9383608b6d6a025434b61e9" exitCode=0 Jul 17 20:03:42.507515 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.507364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" event={"ID":"6055ab65-537a-41a1-87eb-9d9bae4fe02b","Type":"ContainerDied","Data":"059533ec3f3ed785453bd85a511934dd2fdffb40b9383608b6d6a025434b61e9"} Jul 17 20:03:42.508979 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.508950 2578 generic.go:356] "Generic (PLEG): container finished" podID="732066e2-6d11-4713-b407-9404558d7e9b" containerID="f3402c69d9459735b186dd44b64251edaa125681a8ad889b5316f5080a1d7759" exitCode=0 Jul 17 20:03:42.509034 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.508999 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" event={"ID":"732066e2-6d11-4713-b407-9404558d7e9b","Type":"ContainerDied","Data":"f3402c69d9459735b186dd44b64251edaa125681a8ad889b5316f5080a1d7759"} Jul 17 20:03:42.509072 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.509036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" event={"ID":"732066e2-6d11-4713-b407-9404558d7e9b","Type":"ContainerStarted","Data":"5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95"} Jul 17 20:03:42.529660 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.529626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.529808 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.529687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.529808 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.529736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fh5z\" (UniqueName: \"kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.530329 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.530307 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.530427 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.530332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.537282 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.537259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fh5z\" (UniqueName: \"kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.759792 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.759702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:42.895760 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:42.895733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9"] Jul 17 20:03:42.898282 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:03:42.898256 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e1ef89_81e5_4031_8464_626ab305c3b5.slice/crio-d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6 WatchSource:0}: Error finding container d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6: Status 404 returned error can't find the container with id d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6 Jul 17 20:03:43.029903 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.029871 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd"] Jul 17 20:03:43.033472 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.033448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.033857 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.033832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99dtg\" (UniqueName: \"kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.033961 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.033946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.034016 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.033991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.043824 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.043790 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd"] Jul 17 20:03:43.135039 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.135007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.135225 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.135065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99dtg\" (UniqueName: \"kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.135225 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.135157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.135576 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.135501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.135576 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.135514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.154672 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.154643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99dtg\" (UniqueName: \"kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.386104 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.386073 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:43.516957 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.516925 2578 generic.go:356] "Generic (PLEG): container finished" podID="732066e2-6d11-4713-b407-9404558d7e9b" containerID="b8f028d366799e72564e13b68d766a1ec1570237216a8c0cca7a77c4a392f940" exitCode=0 Jul 17 20:03:43.517132 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.517019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" event={"ID":"732066e2-6d11-4713-b407-9404558d7e9b","Type":"ContainerDied","Data":"b8f028d366799e72564e13b68d766a1ec1570237216a8c0cca7a77c4a392f940"} Jul 17 20:03:43.519296 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.519261 2578 generic.go:356] "Generic (PLEG): container finished" podID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerID="a4b1f11de8760c0581122462a41f13bb455d245689ce9c4de9bd7b9494768f9c" exitCode=0 Jul 17 20:03:43.519407 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.519342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" event={"ID":"6055ab65-537a-41a1-87eb-9d9bae4fe02b","Type":"ContainerDied","Data":"a4b1f11de8760c0581122462a41f13bb455d245689ce9c4de9bd7b9494768f9c"} Jul 17 20:03:43.520805 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.520783 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerID="d896b8e07774674dcb006154fea74736de87fe3e904b203314291edd4824a350" exitCode=0 Jul 17 20:03:43.520918 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.520807 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" event={"ID":"e2e1ef89-81e5-4031-8464-626ab305c3b5","Type":"ContainerDied","Data":"d896b8e07774674dcb006154fea74736de87fe3e904b203314291edd4824a350"} Jul 17 20:03:43.520918 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.520840 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" event={"ID":"e2e1ef89-81e5-4031-8464-626ab305c3b5","Type":"ContainerStarted","Data":"d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6"} Jul 17 20:03:43.525809 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:43.525790 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd"] Jul 17 20:03:43.530871 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:03:43.530837 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208959b7_b157_4480_b570_4a7256ca1976.slice/crio-da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846 WatchSource:0}: Error finding container da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846: Status 404 returned error can't find the container with id da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846 Jul 17 20:03:44.528461 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.528431 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerID="6268274ae3651663ee853f39faf14312f248baa5e3c2ae453a3038ba932b5ae0" exitCode=0 Jul 17 20:03:44.528899 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.528512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" event={"ID":"e2e1ef89-81e5-4031-8464-626ab305c3b5","Type":"ContainerDied","Data":"6268274ae3651663ee853f39faf14312f248baa5e3c2ae453a3038ba932b5ae0"} Jul 17 20:03:44.530936 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.530910 2578 generic.go:356] "Generic (PLEG): container finished" podID="732066e2-6d11-4713-b407-9404558d7e9b" containerID="c0a88c1796c0c04edb7b8a7c5eaa750b2086bc4e8c868a5bb17f28220bd90491" exitCode=0 Jul 17 20:03:44.531045 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.530985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" event={"ID":"732066e2-6d11-4713-b407-9404558d7e9b","Type":"ContainerDied","Data":"c0a88c1796c0c04edb7b8a7c5eaa750b2086bc4e8c868a5bb17f28220bd90491"} Jul 17 20:03:44.532679 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.532656 2578 generic.go:356] "Generic (PLEG): container finished" podID="208959b7-b157-4480-b570-4a7256ca1976" containerID="4768d0871e9f862611acba09b38f85a735e1f364dd6de457a6028e5852acbcf5" exitCode=0 Jul 17 20:03:44.532779 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.532719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" event={"ID":"208959b7-b157-4480-b570-4a7256ca1976","Type":"ContainerDied","Data":"4768d0871e9f862611acba09b38f85a735e1f364dd6de457a6028e5852acbcf5"} Jul 17 20:03:44.532779 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.532761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" event={"ID":"208959b7-b157-4480-b570-4a7256ca1976","Type":"ContainerStarted","Data":"da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846"} Jul 17 20:03:44.667492 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.667469 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:44.749873 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.749840 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4d6c\" (UniqueName: \"kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c\") pod \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " Jul 17 20:03:44.750049 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.749910 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util\") pod \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " Jul 17 20:03:44.750049 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.749939 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle\") pod \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\" (UID: \"6055ab65-537a-41a1-87eb-9d9bae4fe02b\") " Jul 17 20:03:44.750401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.750375 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle" (OuterVolumeSpecName: "bundle") pod "6055ab65-537a-41a1-87eb-9d9bae4fe02b" (UID: "6055ab65-537a-41a1-87eb-9d9bae4fe02b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:44.752172 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.752143 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c" (OuterVolumeSpecName: "kube-api-access-l4d6c") pod "6055ab65-537a-41a1-87eb-9d9bae4fe02b" (UID: "6055ab65-537a-41a1-87eb-9d9bae4fe02b"). InnerVolumeSpecName "kube-api-access-l4d6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:03:44.755552 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.755527 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util" (OuterVolumeSpecName: "util") pod "6055ab65-537a-41a1-87eb-9d9bae4fe02b" (UID: "6055ab65-537a-41a1-87eb-9d9bae4fe02b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:44.851467 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.851426 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4d6c\" (UniqueName: \"kubernetes.io/projected/6055ab65-537a-41a1-87eb-9d9bae4fe02b-kube-api-access-l4d6c\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:44.851467 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.851461 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:44.851467 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:44.851472 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6055ab65-537a-41a1-87eb-9d9bae4fe02b-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:45.540163 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.540133 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" Jul 17 20:03:45.540668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.540132 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk" event={"ID":"6055ab65-537a-41a1-87eb-9d9bae4fe02b","Type":"ContainerDied","Data":"257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497"} Jul 17 20:03:45.540668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.540283 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="257a7ae3477a582afe2617217fd7c41dccf5621a1ce1d167a20268e64d440497" Jul 17 20:03:45.542458 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.542416 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerID="19a7da51b9915364acdeae804d71517cb198cbbf2e51a1f62c207264291050e4" exitCode=0 Jul 17 20:03:45.542564 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.542493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" event={"ID":"e2e1ef89-81e5-4031-8464-626ab305c3b5","Type":"ContainerDied","Data":"19a7da51b9915364acdeae804d71517cb198cbbf2e51a1f62c207264291050e4"} Jul 17 20:03:45.707593 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.707391 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:45.760070 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.760042 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util\") pod \"732066e2-6d11-4713-b407-9404558d7e9b\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " Jul 17 20:03:45.760201 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.760083 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfb8d\" (UniqueName: \"kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d\") pod \"732066e2-6d11-4713-b407-9404558d7e9b\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " Jul 17 20:03:45.760201 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.760145 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle\") pod \"732066e2-6d11-4713-b407-9404558d7e9b\" (UID: \"732066e2-6d11-4713-b407-9404558d7e9b\") " Jul 17 20:03:45.760967 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.760934 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle" (OuterVolumeSpecName: "bundle") pod "732066e2-6d11-4713-b407-9404558d7e9b" (UID: "732066e2-6d11-4713-b407-9404558d7e9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:45.762183 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.762159 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d" (OuterVolumeSpecName: "kube-api-access-mfb8d") pod "732066e2-6d11-4713-b407-9404558d7e9b" (UID: "732066e2-6d11-4713-b407-9404558d7e9b"). InnerVolumeSpecName "kube-api-access-mfb8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:03:45.765668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.765643 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util" (OuterVolumeSpecName: "util") pod "732066e2-6d11-4713-b407-9404558d7e9b" (UID: "732066e2-6d11-4713-b407-9404558d7e9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:45.861820 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.861725 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:45.861820 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.861757 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfb8d\" (UniqueName: \"kubernetes.io/projected/732066e2-6d11-4713-b407-9404558d7e9b-kube-api-access-mfb8d\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:45.861820 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:45.861773 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732066e2-6d11-4713-b407-9404558d7e9b-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:46.549712 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.549680 2578 generic.go:356] "Generic (PLEG): container finished" podID="208959b7-b157-4480-b570-4a7256ca1976" containerID="4622d651b348a4c3efc7bb6678089c6617d0c8fd88cc059f31fde4b49065c848" exitCode=0 Jul 17 20:03:46.550136 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.549771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" event={"ID":"208959b7-b157-4480-b570-4a7256ca1976","Type":"ContainerDied","Data":"4622d651b348a4c3efc7bb6678089c6617d0c8fd88cc059f31fde4b49065c848"} Jul 17 20:03:46.551912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.551893 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" Jul 17 20:03:46.552012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.551922 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb" event={"ID":"732066e2-6d11-4713-b407-9404558d7e9b","Type":"ContainerDied","Data":"5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95"} Jul 17 20:03:46.552012 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.551965 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bd620015e0740b62baf34ceea7d7bcbb93a045b97b57c69bc44782a14108e95" Jul 17 20:03:46.693901 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.693850 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:46.769067 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.769031 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle\") pod \"e2e1ef89-81e5-4031-8464-626ab305c3b5\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " Jul 17 20:03:46.769067 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.769072 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fh5z\" (UniqueName: \"kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z\") pod \"e2e1ef89-81e5-4031-8464-626ab305c3b5\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " Jul 17 20:03:46.769322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.769090 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util\") pod \"e2e1ef89-81e5-4031-8464-626ab305c3b5\" (UID: \"e2e1ef89-81e5-4031-8464-626ab305c3b5\") " Jul 17 20:03:46.769539 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.769515 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle" (OuterVolumeSpecName: "bundle") pod "e2e1ef89-81e5-4031-8464-626ab305c3b5" (UID: "e2e1ef89-81e5-4031-8464-626ab305c3b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:46.771243 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.771220 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z" (OuterVolumeSpecName: "kube-api-access-5fh5z") pod "e2e1ef89-81e5-4031-8464-626ab305c3b5" (UID: "e2e1ef89-81e5-4031-8464-626ab305c3b5"). InnerVolumeSpecName "kube-api-access-5fh5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:03:46.774386 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.774360 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util" (OuterVolumeSpecName: "util") pod "e2e1ef89-81e5-4031-8464-626ab305c3b5" (UID: "e2e1ef89-81e5-4031-8464-626ab305c3b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:46.869954 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.869856 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:46.869954 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.869891 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fh5z\" (UniqueName: \"kubernetes.io/projected/e2e1ef89-81e5-4031-8464-626ab305c3b5-kube-api-access-5fh5z\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:46.869954 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:46.869905 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e1ef89-81e5-4031-8464-626ab305c3b5-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:47.559748 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:47.559714 2578 generic.go:356] "Generic (PLEG): container finished" podID="208959b7-b157-4480-b570-4a7256ca1976" containerID="034e71da0babee44eb7902bffff32c92be516a72b4d5b86c4da3edcf076822d9" exitCode=0 Jul 17 20:03:47.560157 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:47.559765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" event={"ID":"208959b7-b157-4480-b570-4a7256ca1976","Type":"ContainerDied","Data":"034e71da0babee44eb7902bffff32c92be516a72b4d5b86c4da3edcf076822d9"} Jul 17 20:03:47.561591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:47.561571 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" Jul 17 20:03:47.561723 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:47.561603 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9" event={"ID":"e2e1ef89-81e5-4031-8464-626ab305c3b5","Type":"ContainerDied","Data":"d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6"} Jul 17 20:03:47.561723 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:47.561653 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d62a1754a9e6740dc7c97548fa312da25fa1a548eb327d9b5f00aa5fa45fcec6" Jul 17 20:03:48.701535 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.701509 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:48.786259 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.786222 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99dtg\" (UniqueName: \"kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg\") pod \"208959b7-b157-4480-b570-4a7256ca1976\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " Jul 17 20:03:48.786463 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.786280 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util\") pod \"208959b7-b157-4480-b570-4a7256ca1976\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " Jul 17 20:03:48.786463 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.786335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle\") pod \"208959b7-b157-4480-b570-4a7256ca1976\" (UID: \"208959b7-b157-4480-b570-4a7256ca1976\") " Jul 17 20:03:48.786872 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.786845 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle" (OuterVolumeSpecName: "bundle") pod "208959b7-b157-4480-b570-4a7256ca1976" (UID: "208959b7-b157-4480-b570-4a7256ca1976"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:48.788374 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.788351 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg" (OuterVolumeSpecName: "kube-api-access-99dtg") pod "208959b7-b157-4480-b570-4a7256ca1976" (UID: "208959b7-b157-4480-b570-4a7256ca1976"). InnerVolumeSpecName "kube-api-access-99dtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:03:48.834603 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.834502 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util" (OuterVolumeSpecName: "util") pod "208959b7-b157-4480-b570-4a7256ca1976" (UID: "208959b7-b157-4480-b570-4a7256ca1976"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:03:48.887800 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.887769 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99dtg\" (UniqueName: \"kubernetes.io/projected/208959b7-b157-4480-b570-4a7256ca1976-kube-api-access-99dtg\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:48.887800 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.887797 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-util\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:48.887925 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:48.887809 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208959b7-b157-4480-b570-4a7256ca1976-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:03:49.573759 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:49.573665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" event={"ID":"208959b7-b157-4480-b570-4a7256ca1976","Type":"ContainerDied","Data":"da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846"} Jul 17 20:03:49.573759 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:49.573685 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd" Jul 17 20:03:49.573759 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:03:49.573737 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da9c540a83cf7fe88bcd6ffb7174a4a9dfb8ad31f021914fad7ee1a58d981846" Jul 17 20:04:15.931025 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:15.930987 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:04:19.186585 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.186547 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb"] Jul 17 20:04:19.187192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187173 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="pull" Jul 17 20:04:19.187192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187194 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="pull" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187218 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="pull" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187226 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="pull" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187235 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="pull" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187245 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="pull" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187262 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187270 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187282 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="util" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187290 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="util" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187308 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187318 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187332 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187342 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="extract" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187356 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="util" Jul 17 20:04:19.187357 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187364 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="util" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187377 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="util" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187385 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="util" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187395 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="util" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187402 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="util" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187412 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="pull" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187419 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="pull" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187429 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="extract" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187436 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="extract" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187521 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="208959b7-b157-4480-b570-4a7256ca1976" containerName="extract" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187535 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e1ef89-81e5-4031-8464-626ab305c3b5" containerName="extract" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187549 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="732066e2-6d11-4713-b407-9404558d7e9b" containerName="extract" Jul 17 20:04:19.188040 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.187560 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6055ab65-537a-41a1-87eb-9d9bae4fe02b" containerName="extract" Jul 17 20:04:19.192598 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.192574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.194820 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.194789 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 20:04:19.194942 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.194822 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 20:04:19.195586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.195567 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 20:04:19.195715 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.195652 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 20:04:19.195715 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.195628 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hf6p\"" Jul 17 20:04:19.198772 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.198748 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb"] Jul 17 20:04:19.370300 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.370267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e0a67f4-aa55-4996-88b8-22acca436193-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.370508 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.370416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xmrl\" (UniqueName: \"kubernetes.io/projected/6e0a67f4-aa55-4996-88b8-22acca436193-kube-api-access-8xmrl\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.370508 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.370452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e0a67f4-aa55-4996-88b8-22acca436193-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.471764 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.471664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xmrl\" (UniqueName: \"kubernetes.io/projected/6e0a67f4-aa55-4996-88b8-22acca436193-kube-api-access-8xmrl\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.471764 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.471711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e0a67f4-aa55-4996-88b8-22acca436193-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.472006 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.471773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e0a67f4-aa55-4996-88b8-22acca436193-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.472448 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.472424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e0a67f4-aa55-4996-88b8-22acca436193-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.474189 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.474171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e0a67f4-aa55-4996-88b8-22acca436193-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.479738 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.479714 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xmrl\" (UniqueName: \"kubernetes.io/projected/6e0a67f4-aa55-4996-88b8-22acca436193-kube-api-access-8xmrl\") pod \"kuadrant-console-plugin-66b5774985-2tddb\" (UID: \"6e0a67f4-aa55-4996-88b8-22acca436193\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.503734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.503697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" Jul 17 20:04:19.646242 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.646213 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb"] Jul 17 20:04:19.648595 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:04:19.648557 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e0a67f4_aa55_4996_88b8_22acca436193.slice/crio-351d9fc559e3b7d38ded806599a760b63e970a7b86b1f15f704c469369b8506c WatchSource:0}: Error finding container 351d9fc559e3b7d38ded806599a760b63e970a7b86b1f15f704c469369b8506c: Status 404 returned error can't find the container with id 351d9fc559e3b7d38ded806599a760b63e970a7b86b1f15f704c469369b8506c Jul 17 20:04:19.730339 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:19.730253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" event={"ID":"6e0a67f4-aa55-4996-88b8-22acca436193","Type":"ContainerStarted","Data":"351d9fc559e3b7d38ded806599a760b63e970a7b86b1f15f704c469369b8506c"} Jul 17 20:04:40.953722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:40.953664 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fc4db4f7b-w4l9f" podUID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" containerName="console" containerID="cri-o://69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae" gracePeriod=15 Jul 17 20:04:42.105069 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.105045 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:04:42.186581 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186549 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btzv8\" (UniqueName: \"kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.186581 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186586 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.186839 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.186839 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186723 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.186839 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186754 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.186839 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186828 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.187048 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.186854 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca\") pod \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\" (UID: \"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9\") " Jul 17 20:04:42.187138 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.187032 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config" (OuterVolumeSpecName: "console-config") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:04:42.187205 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.187170 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:04:42.187391 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.187236 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:04:42.187480 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.187408 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:04:42.188795 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.188774 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:04:42.189236 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.189219 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:04:42.189289 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.189230 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8" (OuterVolumeSpecName: "kube-api-access-btzv8") pod "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" (UID: "03312d05-75a2-4a8a-ab50-db0e7dc6d2d9"). InnerVolumeSpecName "kube-api-access-btzv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:04:42.288394 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288362 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-trusted-ca-bundle\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288394 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288390 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288394 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288399 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btzv8\" (UniqueName: \"kubernetes.io/projected/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-kube-api-access-btzv8\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288408 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288417 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288425 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-console-oauth-config\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.288670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.288435 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9-oauth-serving-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:04:42.879460 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.879424 2578 generic.go:356] "Generic (PLEG): container finished" podID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" containerID="69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae" exitCode=0 Jul 17 20:04:42.879686 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.879471 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc4db4f7b-w4l9f" Jul 17 20:04:42.879686 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.879514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc4db4f7b-w4l9f" event={"ID":"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9","Type":"ContainerDied","Data":"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae"} Jul 17 20:04:42.879686 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.879563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc4db4f7b-w4l9f" event={"ID":"03312d05-75a2-4a8a-ab50-db0e7dc6d2d9","Type":"ContainerDied","Data":"a9d66d80234936bd45ac3de9cc193ff287d903b1efeb62c42c9096943e03a31f"} Jul 17 20:04:42.879686 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.879584 2578 scope.go:117] "RemoveContainer" containerID="69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae" Jul 17 20:04:42.881196 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.881166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" event={"ID":"6e0a67f4-aa55-4996-88b8-22acca436193","Type":"ContainerStarted","Data":"2b55aba76a72f1809c8baacfb1e82a341aace426d95cf644a95d9ecd3b26cde3"} Jul 17 20:04:42.888664 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.888645 2578 scope.go:117] "RemoveContainer" containerID="69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae" Jul 17 20:04:42.888914 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:04:42.888898 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae\": container with ID starting with 69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae not found: ID does not exist" containerID="69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae" Jul 17 20:04:42.888964 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.888921 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae"} err="failed to get container status \"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae\": rpc error: code = NotFound desc = could not find container \"69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae\": container with ID starting with 69400880ee926bf5298cf9b13b153e17eb20b77909f0086da5f93eacad5455ae not found: ID does not exist" Jul 17 20:04:42.903823 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.903798 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:04:42.907133 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.907109 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fc4db4f7b-w4l9f"] Jul 17 20:04:42.923385 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:42.923335 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2tddb" podStartSLOduration=1.557072497 podStartE2EDuration="23.923320623s" totalImagesPullingTime="22.366248126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:04:42.921231407 +0000 UTC m=+518.176462823" watchObservedRunningTime="2026-07-17 20:04:42.923320623 +0000 UTC m=+518.178552039" Jul 17 20:04:43.254596 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:04:43.254563 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" path="/var/lib/kubelet/pods/03312d05-75a2-4a8a-ab50-db0e7dc6d2d9/volumes" Jul 17 20:05:05.867149 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.867110 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:05.867572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.867485 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" containerName="console" Jul 17 20:05:05.867572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.867498 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" containerName="console" Jul 17 20:05:05.867572 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.867562 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="03312d05-75a2-4a8a-ab50-db0e7dc6d2d9" containerName="console" Jul 17 20:05:05.970513 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.970466 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:05.970711 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.970602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:05.972919 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.972897 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l4pjd\"" Jul 17 20:05:05.985779 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.985753 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:05.999017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.998987 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:05.999200 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:05.999069 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:06.098987 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.098955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ttpk\" (UniqueName: \"kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk\") pod \"authorino-75d556bd77-mwqlf\" (UID: \"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5\") " pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:06.098987 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.098995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkw9s\" (UniqueName: \"kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s\") pod \"authorino-58cd955675-584bz\" (UID: \"683ae736-a679-47e4-b806-ebf4f6eca447\") " pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:06.200036 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.200006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ttpk\" (UniqueName: \"kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk\") pod \"authorino-75d556bd77-mwqlf\" (UID: \"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5\") " pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:06.200213 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.200041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkw9s\" (UniqueName: \"kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s\") pod \"authorino-58cd955675-584bz\" (UID: \"683ae736-a679-47e4-b806-ebf4f6eca447\") " pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:06.207553 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.207524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkw9s\" (UniqueName: \"kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s\") pod \"authorino-58cd955675-584bz\" (UID: \"683ae736-a679-47e4-b806-ebf4f6eca447\") " pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:06.207553 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.207549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ttpk\" (UniqueName: \"kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk\") pod \"authorino-75d556bd77-mwqlf\" (UID: \"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5\") " pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:06.280255 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.280221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:06.308543 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.308514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:06.534853 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.534825 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:06.537032 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:05:06.537002 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod683ae736_a679_47e4_b806_ebf4f6eca447.slice/crio-789f3f63d47c08b16a5f1de85f5e23ae0b652c7a3910329fbab5532354c018a7 WatchSource:0}: Error finding container 789f3f63d47c08b16a5f1de85f5e23ae0b652c7a3910329fbab5532354c018a7: Status 404 returned error can't find the container with id 789f3f63d47c08b16a5f1de85f5e23ae0b652c7a3910329fbab5532354c018a7 Jul 17 20:05:06.562300 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:06.562275 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:06.564351 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:05:06.564321 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69bbf0bd_51fc_4151_b8ed_3c2392d7fee5.slice/crio-fa4c892a8257ca0aaf5cacdd3fe9be1a6bcf265e49b1eb6bdcf871ed13d3aac2 WatchSource:0}: Error finding container fa4c892a8257ca0aaf5cacdd3fe9be1a6bcf265e49b1eb6bdcf871ed13d3aac2: Status 404 returned error can't find the container with id fa4c892a8257ca0aaf5cacdd3fe9be1a6bcf265e49b1eb6bdcf871ed13d3aac2 Jul 17 20:05:07.007937 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:07.007902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-584bz" event={"ID":"683ae736-a679-47e4-b806-ebf4f6eca447","Type":"ContainerStarted","Data":"789f3f63d47c08b16a5f1de85f5e23ae0b652c7a3910329fbab5532354c018a7"} Jul 17 20:05:07.009234 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:07.009206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mwqlf" event={"ID":"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5","Type":"ContainerStarted","Data":"fa4c892a8257ca0aaf5cacdd3fe9be1a6bcf265e49b1eb6bdcf871ed13d3aac2"} Jul 17 20:05:12.042414 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:12.042304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-584bz" event={"ID":"683ae736-a679-47e4-b806-ebf4f6eca447","Type":"ContainerStarted","Data":"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599"} Jul 17 20:05:12.043905 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:12.043868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mwqlf" event={"ID":"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5","Type":"ContainerStarted","Data":"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c"} Jul 17 20:05:12.058141 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:12.058092 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-584bz" podStartSLOduration=1.896559356 podStartE2EDuration="7.058076172s" totalImagesPullingTime="5.161516816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:05:12.057145266 +0000 UTC m=+547.312376686" watchObservedRunningTime="2026-07-17 20:05:12.058076172 +0000 UTC m=+547.313307586" Jul 17 20:05:12.072149 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:12.072088 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mwqlf" podStartSLOduration=1.922730485 podStartE2EDuration="7.072068668s" totalImagesPullingTime="5.149338183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:05:12.069038287 +0000 UTC m=+547.324269705" watchObservedRunningTime="2026-07-17 20:05:12.072068668 +0000 UTC m=+547.327300086" Jul 17 20:05:12.092847 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:12.092816 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:14.053625 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:14.053583 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-584bz" podUID="683ae736-a679-47e4-b806-ebf4f6eca447" containerName="authorino" containerID="cri-o://5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599" gracePeriod=30 Jul 17 20:05:14.302007 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:14.301983 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:14.482243 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:14.482203 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkw9s\" (UniqueName: \"kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s\") pod \"683ae736-a679-47e4-b806-ebf4f6eca447\" (UID: \"683ae736-a679-47e4-b806-ebf4f6eca447\") " Jul 17 20:05:14.484252 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:14.484217 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s" (OuterVolumeSpecName: "kube-api-access-rkw9s") pod "683ae736-a679-47e4-b806-ebf4f6eca447" (UID: "683ae736-a679-47e4-b806-ebf4f6eca447"). InnerVolumeSpecName "kube-api-access-rkw9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:05:14.583707 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:14.583668 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkw9s\" (UniqueName: \"kubernetes.io/projected/683ae736-a679-47e4-b806-ebf4f6eca447-kube-api-access-rkw9s\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:05:15.060080 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.060050 2578 generic.go:356] "Generic (PLEG): container finished" podID="683ae736-a679-47e4-b806-ebf4f6eca447" containerID="5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599" exitCode=0 Jul 17 20:05:15.060516 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.060090 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-584bz" Jul 17 20:05:15.060516 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.060103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-584bz" event={"ID":"683ae736-a679-47e4-b806-ebf4f6eca447","Type":"ContainerDied","Data":"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599"} Jul 17 20:05:15.060516 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.060147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-584bz" event={"ID":"683ae736-a679-47e4-b806-ebf4f6eca447","Type":"ContainerDied","Data":"789f3f63d47c08b16a5f1de85f5e23ae0b652c7a3910329fbab5532354c018a7"} Jul 17 20:05:15.060516 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.060167 2578 scope.go:117] "RemoveContainer" containerID="5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599" Jul 17 20:05:15.069247 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.069230 2578 scope.go:117] "RemoveContainer" containerID="5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599" Jul 17 20:05:15.069560 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:05:15.069542 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599\": container with ID starting with 5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599 not found: ID does not exist" containerID="5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599" Jul 17 20:05:15.069649 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.069568 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599"} err="failed to get container status \"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599\": rpc error: code = NotFound desc = could not find container \"5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599\": container with ID starting with 5a38147ddf645d11ba124e92fa48ac6b90668cae74d53bc37f5e10eb67e9e599 not found: ID does not exist" Jul 17 20:05:15.080253 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.080228 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:15.083777 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.083757 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-584bz"] Jul 17 20:05:15.254369 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:15.254336 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683ae736-a679-47e4-b806-ebf4f6eca447" path="/var/lib/kubelet/pods/683ae736-a679-47e4-b806-ebf4f6eca447/volumes" Jul 17 20:05:33.771567 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.771535 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:33.772043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.771901 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683ae736-a679-47e4-b806-ebf4f6eca447" containerName="authorino" Jul 17 20:05:33.772043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.771913 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="683ae736-a679-47e4-b806-ebf4f6eca447" containerName="authorino" Jul 17 20:05:33.772043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.771973 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="683ae736-a679-47e4-b806-ebf4f6eca447" containerName="authorino" Jul 17 20:05:33.774060 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.774044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:33.781983 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.781961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:33.848033 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.848001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcmpb\" (UniqueName: \"kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb\") pod \"authorino-5d4b67c446-2mg8d\" (UID: \"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae\") " pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:33.949194 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.949155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmpb\" (UniqueName: \"kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb\") pod \"authorino-5d4b67c446-2mg8d\" (UID: \"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae\") " pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:33.957237 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.957206 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcmpb\" (UniqueName: \"kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb\") pod \"authorino-5d4b67c446-2mg8d\" (UID: \"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae\") " pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:33.999074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.999042 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:33.999303 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:33.999292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:34.021023 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.020996 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-99765977-5dckh"] Jul 17 20:05:34.024205 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.024121 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.031407 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.031384 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99765977-5dckh"] Jul 17 20:05:34.050659 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.050596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvdnj\" (UniqueName: \"kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj\") pod \"authorino-99765977-5dckh\" (UID: \"f4027455-0162-4baf-a32c-ea080afc9d69\") " pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.135178 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.135153 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:34.137423 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:05:34.137388 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4fe2594_3f0e_4c13_9aa1_b39bba95bfae.slice/crio-a717ee0f47106ddfa6701acb86695b7656b32702e64b8e53496fff0eb3285351 WatchSource:0}: Error finding container a717ee0f47106ddfa6701acb86695b7656b32702e64b8e53496fff0eb3285351: Status 404 returned error can't find the container with id a717ee0f47106ddfa6701acb86695b7656b32702e64b8e53496fff0eb3285351 Jul 17 20:05:34.145925 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.145903 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99765977-5dckh"] Jul 17 20:05:34.146176 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:05:34.146157 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lvdnj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-99765977-5dckh" podUID="f4027455-0162-4baf-a32c-ea080afc9d69" Jul 17 20:05:34.151953 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.151929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvdnj\" (UniqueName: \"kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj\") pod \"authorino-99765977-5dckh\" (UID: \"f4027455-0162-4baf-a32c-ea080afc9d69\") " pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.159966 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.159944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvdnj\" (UniqueName: \"kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj\") pod \"authorino-99765977-5dckh\" (UID: \"f4027455-0162-4baf-a32c-ea080afc9d69\") " pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.161753 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.161721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" event={"ID":"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae","Type":"ContainerStarted","Data":"a717ee0f47106ddfa6701acb86695b7656b32702e64b8e53496fff0eb3285351"} Jul 17 20:05:34.161853 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.161756 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.171569 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.171543 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:05:34.173916 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.173896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:34.174387 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.174369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.176313 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.176293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 17 20:05:34.183379 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.183355 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:05:34.252968 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.252939 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvdnj\" (UniqueName: \"kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj\") pod \"f4027455-0162-4baf-a32c-ea080afc9d69\" (UID: \"f4027455-0162-4baf-a32c-ea080afc9d69\") " Jul 17 20:05:34.253145 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.253130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.253209 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.253181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ccl\" (UniqueName: \"kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.255039 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.255019 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj" (OuterVolumeSpecName: "kube-api-access-lvdnj") pod "f4027455-0162-4baf-a32c-ea080afc9d69" (UID: "f4027455-0162-4baf-a32c-ea080afc9d69"). InnerVolumeSpecName "kube-api-access-lvdnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:05:34.353906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.353814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ccl\" (UniqueName: \"kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.354096 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.353927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.354096 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.353968 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvdnj\" (UniqueName: \"kubernetes.io/projected/f4027455-0162-4baf-a32c-ea080afc9d69-kube-api-access-lvdnj\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:05:34.356337 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.356319 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.360597 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.360582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ccl\" (UniqueName: \"kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl\") pod \"authorino-5cdfbccf57-gm9j5\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.483739 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.483712 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:05:34.626337 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:34.626311 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:05:34.628024 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:05:34.627997 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d943e0f_9c55_4502_9049_e30c9fb59c68.slice/crio-5addeeb71e1c89deec83d9ab6748660176560daa64a92c6a8fe43859e451727d WatchSource:0}: Error finding container 5addeeb71e1c89deec83d9ab6748660176560daa64a92c6a8fe43859e451727d: Status 404 returned error can't find the container with id 5addeeb71e1c89deec83d9ab6748660176560daa64a92c6a8fe43859e451727d Jul 17 20:05:35.176697 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.176651 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" event={"ID":"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae","Type":"ContainerStarted","Data":"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697"} Jul 17 20:05:35.177151 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.176759 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" podUID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" containerName="authorino" containerID="cri-o://aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697" gracePeriod=30 Jul 17 20:05:35.178811 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.178784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" event={"ID":"5d943e0f-9c55-4502-9049-e30c9fb59c68","Type":"ContainerStarted","Data":"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164"} Jul 17 20:05:35.178951 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.178821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" event={"ID":"5d943e0f-9c55-4502-9049-e30c9fb59c68","Type":"ContainerStarted","Data":"5addeeb71e1c89deec83d9ab6748660176560daa64a92c6a8fe43859e451727d"} Jul 17 20:05:35.178951 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.178830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99765977-5dckh" Jul 17 20:05:35.190071 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.190025 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" podStartSLOduration=1.8559295009999999 podStartE2EDuration="2.190010051s" totalImagesPullingTime="334.08055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:05:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:05:35.189034134 +0000 UTC m=+570.444265552" watchObservedRunningTime="2026-07-17 20:05:35.190010051 +0000 UTC m=+570.445241467" Jul 17 20:05:35.208474 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.208430 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" podStartSLOduration=0.75606579 podStartE2EDuration="1.208414325s" totalImagesPullingTime="452.348535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:05:35.206954582 +0000 UTC m=+570.462185998" watchObservedRunningTime="2026-07-17 20:05:35.208414325 +0000 UTC m=+570.463645742" Jul 17 20:05:35.239085 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.239055 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:35.239298 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.239275 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mwqlf" podUID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" containerName="authorino" containerID="cri-o://b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c" gracePeriod=30 Jul 17 20:05:35.258317 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.258266 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99765977-5dckh"] Jul 17 20:05:35.262271 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.262245 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-99765977-5dckh"] Jul 17 20:05:35.492658 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.492635 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:35.503801 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.503777 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:35.565746 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.565715 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcmpb\" (UniqueName: \"kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb\") pod \"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae\" (UID: \"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae\") " Jul 17 20:05:35.565895 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.565775 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ttpk\" (UniqueName: \"kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk\") pod \"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5\" (UID: \"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5\") " Jul 17 20:05:35.567815 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.567785 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk" (OuterVolumeSpecName: "kube-api-access-6ttpk") pod "69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" (UID: "69bbf0bd-51fc-4151-b8ed-3c2392d7fee5"). InnerVolumeSpecName "kube-api-access-6ttpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:05:35.567815 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.567801 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb" (OuterVolumeSpecName: "kube-api-access-tcmpb") pod "f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" (UID: "f4fe2594-3f0e-4c13-9aa1-b39bba95bfae"). InnerVolumeSpecName "kube-api-access-tcmpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:05:35.666591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.666544 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcmpb\" (UniqueName: \"kubernetes.io/projected/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae-kube-api-access-tcmpb\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:05:35.666591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:35.666585 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ttpk\" (UniqueName: \"kubernetes.io/projected/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5-kube-api-access-6ttpk\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:05:36.185668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.185636 2578 generic.go:356] "Generic (PLEG): container finished" podID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" containerID="b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c" exitCode=0 Jul 17 20:05:36.185668 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.185670 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mwqlf" Jul 17 20:05:36.186169 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.185719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mwqlf" event={"ID":"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5","Type":"ContainerDied","Data":"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c"} Jul 17 20:05:36.186169 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.185761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mwqlf" event={"ID":"69bbf0bd-51fc-4151-b8ed-3c2392d7fee5","Type":"ContainerDied","Data":"fa4c892a8257ca0aaf5cacdd3fe9be1a6bcf265e49b1eb6bdcf871ed13d3aac2"} Jul 17 20:05:36.186169 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.185786 2578 scope.go:117] "RemoveContainer" containerID="b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c" Jul 17 20:05:36.187249 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.187230 2578 generic.go:356] "Generic (PLEG): container finished" podID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" containerID="aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697" exitCode=0 Jul 17 20:05:36.187350 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.187281 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" Jul 17 20:05:36.187410 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.187263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" event={"ID":"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae","Type":"ContainerDied","Data":"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697"} Jul 17 20:05:36.187410 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.187389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2mg8d" event={"ID":"f4fe2594-3f0e-4c13-9aa1-b39bba95bfae","Type":"ContainerDied","Data":"a717ee0f47106ddfa6701acb86695b7656b32702e64b8e53496fff0eb3285351"} Jul 17 20:05:36.195015 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.194999 2578 scope.go:117] "RemoveContainer" containerID="b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c" Jul 17 20:05:36.195333 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:05:36.195316 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c\": container with ID starting with b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c not found: ID does not exist" containerID="b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c" Jul 17 20:05:36.195382 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.195342 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c"} err="failed to get container status \"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c\": rpc error: code = NotFound desc = could not find container \"b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c\": container with ID starting with b8f289b33247d3858ae8108ef000e65d92e1fa3b860d23f4fa55279185ee266c not found: ID does not exist" Jul 17 20:05:36.195382 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.195360 2578 scope.go:117] "RemoveContainer" containerID="aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697" Jul 17 20:05:36.204240 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.204180 2578 scope.go:117] "RemoveContainer" containerID="aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697" Jul 17 20:05:36.204533 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:05:36.204507 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697\": container with ID starting with aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697 not found: ID does not exist" containerID="aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697" Jul 17 20:05:36.204659 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.204539 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697"} err="failed to get container status \"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697\": rpc error: code = NotFound desc = could not find container \"aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697\": container with ID starting with aa52500b84b61cd0e83d8a4a8694f1301618775f33c4998dcd7904735be19697 not found: ID does not exist" Jul 17 20:05:36.206518 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.206500 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:36.209999 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.209975 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mwqlf"] Jul 17 20:05:36.218579 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.218514 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:36.220594 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:36.220575 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2mg8d"] Jul 17 20:05:37.253741 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:37.253707 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" path="/var/lib/kubelet/pods/69bbf0bd-51fc-4151-b8ed-3c2392d7fee5/volumes" Jul 17 20:05:37.254119 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:37.254042 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4027455-0162-4baf-a32c-ea080afc9d69" path="/var/lib/kubelet/pods/f4027455-0162-4baf-a32c-ea080afc9d69/volumes" Jul 17 20:05:37.254232 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:05:37.254219 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" path="/var/lib/kubelet/pods/f4fe2594-3f0e-4c13-9aa1-b39bba95bfae/volumes" Jul 17 20:06:05.178605 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:05.178565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:06:05.180087 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:05.180066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:06:18.174958 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.174922 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q77xz"] Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175259 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" containerName="authorino" Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175270 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" containerName="authorino" Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175281 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" containerName="authorino" Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175288 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" containerName="authorino" Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175376 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69bbf0bd-51fc-4151-b8ed-3c2392d7fee5" containerName="authorino" Jul 17 20:06:18.177433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.175388 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4fe2594-3f0e-4c13-9aa1-b39bba95bfae" containerName="authorino" Jul 17 20:06:18.178391 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.178375 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.180555 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.180531 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 17 20:06:18.180694 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.180581 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2xtl7\"" Jul 17 20:06:18.187220 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.187193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q77xz"] Jul 17 20:06:18.239384 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.239349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5acdaa-3ae6-4d88-8525-866c2de72469-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.239550 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.239429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62vxv\" (UniqueName: \"kubernetes.io/projected/bb5acdaa-3ae6-4d88-8525-866c2de72469-kube-api-access-62vxv\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.340863 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.340828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62vxv\" (UniqueName: \"kubernetes.io/projected/bb5acdaa-3ae6-4d88-8525-866c2de72469-kube-api-access-62vxv\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.341027 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.340974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5acdaa-3ae6-4d88-8525-866c2de72469-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.341594 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.341575 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5acdaa-3ae6-4d88-8525-866c2de72469-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.355669 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.355637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62vxv\" (UniqueName: \"kubernetes.io/projected/bb5acdaa-3ae6-4d88-8525-866c2de72469-kube-api-access-62vxv\") pod \"payload-processing-5bf47cd988-q77xz\" (UID: \"bb5acdaa-3ae6-4d88-8525-866c2de72469\") " pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.490294 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.490195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:18.624583 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.624562 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q77xz"] Jul 17 20:06:18.626093 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:06:18.626066 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5acdaa_3ae6_4d88_8525_866c2de72469.slice/crio-44f5acef93bca65fbdabfae9d5cafc97fbc1b8cf667d3a8ab05c87313200f100 WatchSource:0}: Error finding container 44f5acef93bca65fbdabfae9d5cafc97fbc1b8cf667d3a8ab05c87313200f100: Status 404 returned error can't find the container with id 44f5acef93bca65fbdabfae9d5cafc97fbc1b8cf667d3a8ab05c87313200f100 Jul 17 20:06:18.628446 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:18.628428 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 20:06:19.421626 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:19.421568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" event={"ID":"bb5acdaa-3ae6-4d88-8525-866c2de72469","Type":"ContainerStarted","Data":"44f5acef93bca65fbdabfae9d5cafc97fbc1b8cf667d3a8ab05c87313200f100"} Jul 17 20:06:20.429424 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:20.429384 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" event={"ID":"bb5acdaa-3ae6-4d88-8525-866c2de72469","Type":"ContainerStarted","Data":"aeace0837847c1524c1c408ca61447b3514c3d5bdf38be29af907e5f72a2a28f"} Jul 17 20:06:20.429912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:20.429560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:20.445543 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:20.445493 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" podStartSLOduration=0.871814588 podStartE2EDuration="2.445479543s" totalImagesPullingTime="1.573664955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:06:20.443018267 +0000 UTC m=+615.698249684" watchObservedRunningTime="2026-07-17 20:06:20.445479543 +0000 UTC m=+615.700710960" Jul 17 20:06:31.438678 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:31.438581 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-q77xz" Jul 17 20:06:54.076709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.076663 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m"] Jul 17 20:06:54.083150 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.083128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.085443 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.085419 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 20:06:54.085577 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.085545 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pjnw2\"" Jul 17 20:06:54.085653 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.085594 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 20:06:54.086129 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.086115 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 17 20:06:54.092196 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.092176 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m"] Jul 17 20:06:54.269269 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.269269 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.269506 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8736cfab-fc84-4db8-9759-0269829b0d51-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.269506 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269440 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.269506 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.269651 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.269505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-567bw\" (UniqueName: \"kubernetes.io/projected/8736cfab-fc84-4db8-9759-0269829b0d51-kube-api-access-567bw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.370835 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8736cfab-fc84-4db8-9759-0269829b0d51-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.370835 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.370835 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.370835 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-567bw\" (UniqueName: \"kubernetes.io/projected/8736cfab-fc84-4db8-9759-0269829b0d51-kube-api-access-567bw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.371177 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.371177 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.370926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.371330 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.371308 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.371385 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.371348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.371441 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.371383 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.373121 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.373098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8736cfab-fc84-4db8-9759-0269829b0d51-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.373449 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.373430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8736cfab-fc84-4db8-9759-0269829b0d51-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.378540 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.378519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-567bw\" (UniqueName: \"kubernetes.io/projected/8736cfab-fc84-4db8-9759-0269829b0d51-kube-api-access-567bw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqr6m\" (UID: \"8736cfab-fc84-4db8-9759-0269829b0d51\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.395471 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.395444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:06:54.529857 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.529833 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m"] Jul 17 20:06:54.531283 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:06:54.531255 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8736cfab_fc84_4db8_9759_0269829b0d51.slice/crio-c4369aa241d735f86c41b257d02d43fe96ebca89796845db0987623e076b0c47 WatchSource:0}: Error finding container c4369aa241d735f86c41b257d02d43fe96ebca89796845db0987623e076b0c47: Status 404 returned error can't find the container with id c4369aa241d735f86c41b257d02d43fe96ebca89796845db0987623e076b0c47 Jul 17 20:06:54.607467 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:54.607433 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" event={"ID":"8736cfab-fc84-4db8-9759-0269829b0d51","Type":"ContainerStarted","Data":"c4369aa241d735f86c41b257d02d43fe96ebca89796845db0987623e076b0c47"} Jul 17 20:06:56.620838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:56.620804 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" event={"ID":"8736cfab-fc84-4db8-9759-0269829b0d51","Type":"ContainerStarted","Data":"90f7b7deef2e451609be2786ee65bbfff76b2663b1feedf217720652c28981df"} Jul 17 20:06:56.638658 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:06:56.638592 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" podStartSLOduration=1.59379944 podStartE2EDuration="2.638577097s" totalImagesPullingTime="1.044777657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:06:56.636264766 +0000 UTC m=+651.891496184" watchObservedRunningTime="2026-07-17 20:06:56.638577097 +0000 UTC m=+651.893808518" Jul 17 20:07:04.395940 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:04.395896 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:07:04.396305 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:04.395988 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:07:04.408465 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:04.408441 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:07:04.674085 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:04.674002 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqr6m" Jul 17 20:07:30.375255 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.375221 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th"] Jul 17 20:07:30.379059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.379044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.381169 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.381145 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 17 20:07:30.387429 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.387406 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th"] Jul 17 20:07:30.501511 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.501722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.501722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.501722 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33853ccb-d364-4980-8158-0d3b10acf11d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.501874 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hskk\" (UniqueName: \"kubernetes.io/projected/33853ccb-d364-4980-8158-0d3b10acf11d-kube-api-access-2hskk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.501874 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.501755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602370 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33853ccb-d364-4980-8158-0d3b10acf11d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hskk\" (UniqueName: \"kubernetes.io/projected/33853ccb-d364-4980-8158-0d3b10acf11d-kube-api-access-2hskk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602586 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602977 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602952 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.602977 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.602975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.603128 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.603027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.604705 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.604684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33853ccb-d364-4980-8158-0d3b10acf11d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.604872 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.604857 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33853ccb-d364-4980-8158-0d3b10acf11d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.610027 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.610007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hskk\" (UniqueName: \"kubernetes.io/projected/33853ccb-d364-4980-8158-0d3b10acf11d-kube-api-access-2hskk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc5th\" (UID: \"33853ccb-d364-4980-8158-0d3b10acf11d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.690199 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.690167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:30.825624 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:30.825585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th"] Jul 17 20:07:30.827056 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:07:30.827031 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33853ccb_d364_4980_8158_0d3b10acf11d.slice/crio-9aca705dd12c252bf268ae6d814230793768e8ce85a0e17d9e9d84453f32d64e WatchSource:0}: Error finding container 9aca705dd12c252bf268ae6d814230793768e8ce85a0e17d9e9d84453f32d64e: Status 404 returned error can't find the container with id 9aca705dd12c252bf268ae6d814230793768e8ce85a0e17d9e9d84453f32d64e Jul 17 20:07:31.807785 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:31.807745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" event={"ID":"33853ccb-d364-4980-8158-0d3b10acf11d","Type":"ContainerStarted","Data":"8962819b3eedd6505a67f3eda036d5d5f93584082c5ee071aef69a3897584bab"} Jul 17 20:07:31.808185 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:31.807794 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" event={"ID":"33853ccb-d364-4980-8158-0d3b10acf11d","Type":"ContainerStarted","Data":"9aca705dd12c252bf268ae6d814230793768e8ce85a0e17d9e9d84453f32d64e"} Jul 17 20:07:31.829942 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:31.827044 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" podStartSLOduration=1.612689343 podStartE2EDuration="1.827024358s" totalImagesPullingTime="214.335015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:07:31.823940581 +0000 UTC m=+687.079172000" watchObservedRunningTime="2026-07-17 20:07:31.827024358 +0000 UTC m=+687.082255777" Jul 17 20:07:40.691318 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:40.691280 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:40.691735 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:40.691376 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:40.704457 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:40.704432 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:40.864896 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:40.864860 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc5th" Jul 17 20:07:44.580833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.580799 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7"] Jul 17 20:07:44.584978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.584958 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.587249 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.587220 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 17 20:07:44.594312 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.594286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7"] Jul 17 20:07:44.735363 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.735556 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735412 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.735556 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.735556 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede1a23a-0c24-4af8-83bf-90b038876e63-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.735556 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.735757 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.735646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdbnq\" (UniqueName: \"kubernetes.io/projected/ede1a23a-0c24-4af8-83bf-90b038876e63-kube-api-access-gdbnq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836578 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836578 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdbnq\" (UniqueName: \"kubernetes.io/projected/ede1a23a-0c24-4af8-83bf-90b038876e63-kube-api-access-gdbnq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836578 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836797 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede1a23a-0c24-4af8-83bf-90b038876e63-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.836994 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.836975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.837057 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.837043 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.837115 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.837073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.838960 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.838938 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede1a23a-0c24-4af8-83bf-90b038876e63-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.839183 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.839166 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede1a23a-0c24-4af8-83bf-90b038876e63-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.843342 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.843320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdbnq\" (UniqueName: \"kubernetes.io/projected/ede1a23a-0c24-4af8-83bf-90b038876e63-kube-api-access-gdbnq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7\" (UID: \"ede1a23a-0c24-4af8-83bf-90b038876e63\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:44.896813 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:44.896782 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:45.031625 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:45.031581 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7"] Jul 17 20:07:45.032593 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:07:45.032571 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podede1a23a_0c24_4af8_83bf_90b038876e63.slice/crio-9a1bd7e99a88cac93e704d44594a5af6f19aafcdf5d33d90d293d0d73e673fa1 WatchSource:0}: Error finding container 9a1bd7e99a88cac93e704d44594a5af6f19aafcdf5d33d90d293d0d73e673fa1: Status 404 returned error can't find the container with id 9a1bd7e99a88cac93e704d44594a5af6f19aafcdf5d33d90d293d0d73e673fa1 Jul 17 20:07:45.879581 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:45.879547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" event={"ID":"ede1a23a-0c24-4af8-83bf-90b038876e63","Type":"ContainerStarted","Data":"54d5f37fe98a910006ee89105127a916b07ed5be307d8489a48230bd8de88a3f"} Jul 17 20:07:45.879581 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:45.879592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" event={"ID":"ede1a23a-0c24-4af8-83bf-90b038876e63","Type":"ContainerStarted","Data":"9a1bd7e99a88cac93e704d44594a5af6f19aafcdf5d33d90d293d0d73e673fa1"} Jul 17 20:07:45.894739 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:45.894687 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" podStartSLOduration=1.716708522 podStartE2EDuration="1.894665918s" totalImagesPullingTime="177.957396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:07:45.894151614 +0000 UTC m=+701.149383045" watchObservedRunningTime="2026-07-17 20:07:45.894665918 +0000 UTC m=+701.149897336" Jul 17 20:07:54.897534 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:54.897483 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:54.897534 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:54.897543 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:54.910229 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:54.910202 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:07:54.939059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:07:54.939035 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7" Jul 17 20:09:01.835009 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:01.834971 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-664d5b5dd4-j2nf7"] Jul 17 20:09:01.838348 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:01.838330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:01.845043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:01.845012 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664d5b5dd4-j2nf7"] Jul 17 20:09:01.921400 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:01.921361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/500a8d18-8b09-41f9-98e6-d7e376c3b820-tls-cert\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:01.921400 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:01.921399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52lpk\" (UniqueName: \"kubernetes.io/projected/500a8d18-8b09-41f9-98e6-d7e376c3b820-kube-api-access-52lpk\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.022565 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.022519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/500a8d18-8b09-41f9-98e6-d7e376c3b820-tls-cert\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.022565 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.022572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52lpk\" (UniqueName: \"kubernetes.io/projected/500a8d18-8b09-41f9-98e6-d7e376c3b820-kube-api-access-52lpk\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.025034 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.025012 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/500a8d18-8b09-41f9-98e6-d7e376c3b820-tls-cert\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.029972 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.029953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52lpk\" (UniqueName: \"kubernetes.io/projected/500a8d18-8b09-41f9-98e6-d7e376c3b820-kube-api-access-52lpk\") pod \"authorino-664d5b5dd4-j2nf7\" (UID: \"500a8d18-8b09-41f9-98e6-d7e376c3b820\") " pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.149577 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.149547 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" Jul 17 20:09:02.313891 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:02.313866 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664d5b5dd4-j2nf7"] Jul 17 20:09:02.315604 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:09:02.315567 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod500a8d18_8b09_41f9_98e6_d7e376c3b820.slice/crio-4a015879425d34429b8aa79d42be762939fb5c7fc269399e94f1979e25a73d4d WatchSource:0}: Error finding container 4a015879425d34429b8aa79d42be762939fb5c7fc269399e94f1979e25a73d4d: Status 404 returned error can't find the container with id 4a015879425d34429b8aa79d42be762939fb5c7fc269399e94f1979e25a73d4d Jul 17 20:09:03.305388 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.305342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" event={"ID":"500a8d18-8b09-41f9-98e6-d7e376c3b820","Type":"ContainerStarted","Data":"d7dc477ed889f07e0d2aa10d1e83168a835a478a31582a26810cfef966575dd8"} Jul 17 20:09:03.305849 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.305406 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" event={"ID":"500a8d18-8b09-41f9-98e6-d7e376c3b820","Type":"ContainerStarted","Data":"4a015879425d34429b8aa79d42be762939fb5c7fc269399e94f1979e25a73d4d"} Jul 17 20:09:03.324923 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.324866 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-664d5b5dd4-j2nf7" podStartSLOduration=1.9516510889999998 podStartE2EDuration="2.3248481s" totalImagesPullingTime="373.197011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:09:03.323452775 +0000 UTC m=+778.578684193" watchObservedRunningTime="2026-07-17 20:09:03.3248481 +0000 UTC m=+778.580079518" Jul 17 20:09:03.357117 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.357074 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:09:03.357601 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.357336 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" podUID="5d943e0f-9c55-4502-9049-e30c9fb59c68" containerName="authorino" containerID="cri-o://91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164" gracePeriod=30 Jul 17 20:09:03.636729 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.636675 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:09:03.741914 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.741880 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert\") pod \"5d943e0f-9c55-4502-9049-e30c9fb59c68\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " Jul 17 20:09:03.742191 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.742002 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4ccl\" (UniqueName: \"kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl\") pod \"5d943e0f-9c55-4502-9049-e30c9fb59c68\" (UID: \"5d943e0f-9c55-4502-9049-e30c9fb59c68\") " Jul 17 20:09:03.744011 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.743977 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl" (OuterVolumeSpecName: "kube-api-access-j4ccl") pod "5d943e0f-9c55-4502-9049-e30c9fb59c68" (UID: "5d943e0f-9c55-4502-9049-e30c9fb59c68"). InnerVolumeSpecName "kube-api-access-j4ccl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:09:03.752990 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.752961 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5d943e0f-9c55-4502-9049-e30c9fb59c68" (UID: "5d943e0f-9c55-4502-9049-e30c9fb59c68"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:09:03.843655 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.843593 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4ccl\" (UniqueName: \"kubernetes.io/projected/5d943e0f-9c55-4502-9049-e30c9fb59c68-kube-api-access-j4ccl\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:09:03.843655 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:03.843650 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d943e0f-9c55-4502-9049-e30c9fb59c68-tls-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:09:04.326126 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.326028 2578 generic.go:356] "Generic (PLEG): container finished" podID="5d943e0f-9c55-4502-9049-e30c9fb59c68" containerID="91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164" exitCode=0 Jul 17 20:09:04.330633 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.326725 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" Jul 17 20:09:04.330633 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.329674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" event={"ID":"5d943e0f-9c55-4502-9049-e30c9fb59c68","Type":"ContainerDied","Data":"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164"} Jul 17 20:09:04.330633 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.329730 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdfbccf57-gm9j5" event={"ID":"5d943e0f-9c55-4502-9049-e30c9fb59c68","Type":"ContainerDied","Data":"5addeeb71e1c89deec83d9ab6748660176560daa64a92c6a8fe43859e451727d"} Jul 17 20:09:04.330633 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.329754 2578 scope.go:117] "RemoveContainer" containerID="91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164" Jul 17 20:09:04.345525 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.345498 2578 scope.go:117] "RemoveContainer" containerID="91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164" Jul 17 20:09:04.345957 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:09:04.345931 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164\": container with ID starting with 91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164 not found: ID does not exist" containerID="91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164" Jul 17 20:09:04.346071 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.345965 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164"} err="failed to get container status \"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164\": rpc error: code = NotFound desc = could not find container \"91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164\": container with ID starting with 91ba1069c2d43e406b88ea8074b6178c61ef09b2965f79351290e28d31b02164 not found: ID does not exist" Jul 17 20:09:04.362555 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.362512 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:09:04.365683 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:04.365655 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cdfbccf57-gm9j5"] Jul 17 20:09:05.255751 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:09:05.255728 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d943e0f-9c55-4502-9049-e30c9fb59c68" path="/var/lib/kubelet/pods/5d943e0f-9c55-4502-9049-e30c9fb59c68/volumes" Jul 17 20:11:05.216590 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:11:05.216519 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:11:05.216590 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:11:05.216574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:16:05.254356 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:16:05.254275 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:16:05.257032 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:16:05.257010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:21:05.292829 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:21:05.292798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:21:05.298875 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:21:05.298846 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:26:05.333035 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:26:05.333009 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:26:05.340085 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:26:05.340059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:31:05.369713 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:31:05.369634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:31:05.385111 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:31:05.385085 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:36:05.415887 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:36:05.415858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:36:05.424707 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:36:05.424682 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:40:07.382590 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.382510 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:40:07.383177 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.383080 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d943e0f-9c55-4502-9049-e30c9fb59c68" containerName="authorino" Jul 17 20:40:07.383177 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.383103 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d943e0f-9c55-4502-9049-e30c9fb59c68" containerName="authorino" Jul 17 20:40:07.383324 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.383217 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d943e0f-9c55-4502-9049-e30c9fb59c68" containerName="authorino" Jul 17 20:40:07.386356 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.386338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.389827 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.389803 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6d55d2b6\"/\"kube-root-ca.crt\"" Jul 17 20:40:07.390073 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.389806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6d55d2b6\"/\"e2e-lifecycle-model-6d55d2b6-kserve-self-signed-certs\"" Jul 17 20:40:07.390183 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.389819 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6d55d2b6\"/\"default-dockercfg-hfkxs\"" Jul 17 20:40:07.390183 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.389835 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6d55d2b6\"/\"openshift-service-ca.crt\"" Jul 17 20:40:07.396884 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.396859 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:40:07.532955 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.532911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spstr\" (UniqueName: \"kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.532955 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.532956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.533184 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.532983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.533184 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.533084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.533184 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.533112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.533184 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.533172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.633924 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.633827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spstr\" (UniqueName: \"kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.633924 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.633875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.633924 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.633907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634163 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.633965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634163 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.633988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634163 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.634049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634392 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.634371 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634445 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.634422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.634492 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.634454 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.636185 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.636162 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.636490 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.636472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.641093 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.641066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spstr\" (UniqueName: \"kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr\") pod \"e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.697390 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.697352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:07.835268 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:07.835240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:40:07.837163 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:40:07.837132 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02037047_83af_4319_b640_c836585ac3f8.slice/crio-fa24829ddf2878c280b81a93074ea3b79a656966a01ef8963cf5d6d4df37e06c WatchSource:0}: Error finding container fa24829ddf2878c280b81a93074ea3b79a656966a01ef8963cf5d6d4df37e06c: Status 404 returned error can't find the container with id fa24829ddf2878c280b81a93074ea3b79a656966a01ef8963cf5d6d4df37e06c Jul 17 20:40:08.573757 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:08.573722 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" event={"ID":"02037047-83af-4319-b640-c836585ac3f8","Type":"ContainerStarted","Data":"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84"} Jul 17 20:40:08.573757 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:08.573769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" event={"ID":"02037047-83af-4319-b640-c836585ac3f8","Type":"ContainerStarted","Data":"fa24829ddf2878c280b81a93074ea3b79a656966a01ef8963cf5d6d4df37e06c"} Jul 17 20:40:08.594084 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:08.594030 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" podStartSLOduration=1.594011014 podStartE2EDuration="1.594011014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:40:08.591072218 +0000 UTC m=+2643.846303636" watchObservedRunningTime="2026-07-17 20:40:08.594011014 +0000 UTC m=+2643.849242431" Jul 17 20:40:17.698118 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:17.698076 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:17.698118 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:17.698125 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:17.710796 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:17.710766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:18.637293 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:18.637259 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:40:49.152414 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:40:49.152367 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:41:04.173957 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.173856 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" podUID="02037047-83af-4319-b640-c836585ac3f8" containerName="main" containerID="cri-o://953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84" gracePeriod=45 Jul 17 20:41:04.426015 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.425951 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:41:04.471205 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471172 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471286 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471311 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471334 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471386 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spstr\" (UniqueName: \"kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471638 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471425 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir\") pod \"02037047-83af-4319-b640-c836585ac3f8\" (UID: \"02037047-83af-4319-b640-c836585ac3f8\") " Jul 17 20:41:04.471638 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471500 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home" (OuterVolumeSpecName: "home") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:41:04.471781 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471746 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:41:04.471836 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.471803 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.472095 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.472066 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:41:04.473554 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.473523 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:41:04.473713 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.473681 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr" (OuterVolumeSpecName: "kube-api-access-spstr") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "kube-api-access-spstr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:41:04.473853 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.473835 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm" (OuterVolumeSpecName: "dshm") pod "02037047-83af-4319-b640-c836585ac3f8" (UID: "02037047-83af-4319-b640-c836585ac3f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:41:04.572906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.572860 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02037047-83af-4319-b640-c836585ac3f8-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.572906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.572896 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.572906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.572906 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.572906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.572915 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spstr\" (UniqueName: \"kubernetes.io/projected/02037047-83af-4319-b640-c836585ac3f8-kube-api-access-spstr\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.573181 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.572926 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02037047-83af-4319-b640-c836585ac3f8-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:41:04.882630 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.882510 2578 generic.go:356] "Generic (PLEG): container finished" podID="02037047-83af-4319-b640-c836585ac3f8" containerID="953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84" exitCode=0 Jul 17 20:41:04.882630 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.882563 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" Jul 17 20:41:04.882630 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.882598 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" event={"ID":"02037047-83af-4319-b640-c836585ac3f8","Type":"ContainerDied","Data":"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84"} Jul 17 20:41:04.882914 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.882665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw" event={"ID":"02037047-83af-4319-b640-c836585ac3f8","Type":"ContainerDied","Data":"fa24829ddf2878c280b81a93074ea3b79a656966a01ef8963cf5d6d4df37e06c"} Jul 17 20:41:04.882914 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.882688 2578 scope.go:117] "RemoveContainer" containerID="953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84" Jul 17 20:41:04.891949 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.891931 2578 scope.go:117] "RemoveContainer" containerID="953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84" Jul 17 20:41:04.892213 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:41:04.892196 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84\": container with ID starting with 953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84 not found: ID does not exist" containerID="953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84" Jul 17 20:41:04.892263 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.892220 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84"} err="failed to get container status \"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84\": rpc error: code = NotFound desc = could not find container \"953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84\": container with ID starting with 953f0733f070a9f3b50052f9704394770d1a7ccc8326e53b34530875bef4df84 not found: ID does not exist" Jul 17 20:41:04.904463 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.904422 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:41:04.907590 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:04.907559 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6d55d2b6/e2e-lifecycle-model-6d55d2b6-kserve-c694589cf-hswjw"] Jul 17 20:41:05.254000 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:05.253968 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02037047-83af-4319-b640-c836585ac3f8" path="/var/lib/kubelet/pods/02037047-83af-4319-b640-c836585ac3f8/volumes" Jul 17 20:41:05.451936 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:05.451907 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:41:05.461185 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:41:05.461160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:43:21.438789 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.438754 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:21.439435 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.439140 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02037047-83af-4319-b640-c836585ac3f8" containerName="main" Jul 17 20:43:21.439435 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.439152 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="02037047-83af-4319-b640-c836585ac3f8" containerName="main" Jul 17 20:43:21.439435 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.439212 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="02037047-83af-4319-b640-c836585ac3f8" containerName="main" Jul 17 20:43:21.442263 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.442246 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:21.444694 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.444670 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kc768\"" Jul 17 20:43:21.444838 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.444736 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 20:43:21.451822 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.451797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:21.624205 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.624173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-t8d2k\" (UID: \"500a86b5-d0f0-4e51-876d-289583edc9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:21.725742 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.725649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-t8d2k\" (UID: \"500a86b5-d0f0-4e51-876d-289583edc9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:21.726333 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.726305 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-t8d2k\" (UID: \"500a86b5-d0f0-4e51-876d-289583edc9b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:21.753768 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.753735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:21.892358 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.892336 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:21.893670 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:43:21.893645 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500a86b5_d0f0_4e51_876d_289583edc9b2.slice/crio-8c5f40b1c4cb3b7fe39ab8332780e70c8f2aa19c879928f0970f0aa64baf4ad1 WatchSource:0}: Error finding container 8c5f40b1c4cb3b7fe39ab8332780e70c8f2aa19c879928f0970f0aa64baf4ad1: Status 404 returned error can't find the container with id 8c5f40b1c4cb3b7fe39ab8332780e70c8f2aa19c879928f0970f0aa64baf4ad1 Jul 17 20:43:21.895457 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:21.895442 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 20:43:22.635121 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:22.635086 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerStarted","Data":"8c5f40b1c4cb3b7fe39ab8332780e70c8f2aa19c879928f0970f0aa64baf4ad1"} Jul 17 20:43:25.655428 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:25.655392 2578 generic.go:356] "Generic (PLEG): container finished" podID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerID="1a3bee004d5fd970f6ed5f6e6d54120be985009bbd8be7471e7a6aa907c5e29c" exitCode=6 Jul 17 20:43:25.655912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:25.655451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerDied","Data":"1a3bee004d5fd970f6ed5f6e6d54120be985009bbd8be7471e7a6aa907c5e29c"} Jul 17 20:43:25.655912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:25.655788 2578 scope.go:117] "RemoveContainer" containerID="1a3bee004d5fd970f6ed5f6e6d54120be985009bbd8be7471e7a6aa907c5e29c" Jul 17 20:43:26.663318 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:26.663289 2578 generic.go:356] "Generic (PLEG): container finished" podID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerID="f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078" exitCode=6 Jul 17 20:43:26.663744 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:26.663347 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerDied","Data":"f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078"} Jul 17 20:43:26.663744 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:26.663394 2578 scope.go:117] "RemoveContainer" containerID="1a3bee004d5fd970f6ed5f6e6d54120be985009bbd8be7471e7a6aa907c5e29c" Jul 17 20:43:26.663744 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:26.663533 2578 scope.go:117] "RemoveContainer" containerID="f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078" Jul 17 20:43:26.663879 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:43:26.663804 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-t8d2k_odh-ai-gateway-infra(500a86b5-d0f0-4e51-876d-289583edc9b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" Jul 17 20:43:27.670136 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:27.670109 2578 scope.go:117] "RemoveContainer" containerID="f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078" Jul 17 20:43:27.670540 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:43:27.670322 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-t8d2k_odh-ai-gateway-infra(500a86b5-d0f0-4e51-876d-289583edc9b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" Jul 17 20:43:28.884434 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.884400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:43:28.889533 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.889511 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.892948 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.892925 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f0f4f\"/\"kube-root-ca.crt\"" Jul 17 20:43:28.893329 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.893305 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f0f4f\"/\"test-model-5f0f4f-kserve-self-signed-certs\"" Jul 17 20:43:28.893476 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.893327 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f0f4f\"/\"default-dockercfg-zxhjp\"" Jul 17 20:43:28.893476 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.893401 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f0f4f\"/\"openshift-service-ca.crt\"" Jul 17 20:43:28.900728 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.900705 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:43:28.989933 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.989892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.989933 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.989939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.990179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.989961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktj7s\" (UniqueName: \"kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.990179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.990049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.990179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.990070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:28.990179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:28.990084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091008 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.090972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091008 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091216 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091216 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091216 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktj7s\" (UniqueName: \"kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091216 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091505 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091480 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091564 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.091564 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.091534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.093404 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.093373 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.093667 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.093650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.099716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.099692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktj7s\" (UniqueName: \"kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s\") pod \"test-model-5f0f4f-kserve-54bff96f55-cdgdk\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.200523 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.200482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:29.343485 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.343452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:43:29.344742 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:43:29.344716 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99c1c8f1_f8f9_4454_b8a0_279722612fa2.slice/crio-60f86ea263a7af605f7f051758f5a3edff182a80e57eb05f5de693b18abb9146 WatchSource:0}: Error finding container 60f86ea263a7af605f7f051758f5a3edff182a80e57eb05f5de693b18abb9146: Status 404 returned error can't find the container with id 60f86ea263a7af605f7f051758f5a3edff182a80e57eb05f5de693b18abb9146 Jul 17 20:43:29.687106 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.687070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" event={"ID":"99c1c8f1-f8f9-4454-b8a0-279722612fa2","Type":"ContainerStarted","Data":"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803"} Jul 17 20:43:29.687106 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.687111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" event={"ID":"99c1c8f1-f8f9-4454-b8a0-279722612fa2","Type":"ContainerStarted","Data":"60f86ea263a7af605f7f051758f5a3edff182a80e57eb05f5de693b18abb9146"} Jul 17 20:43:29.705923 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:29.705869 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" podStartSLOduration=1.705849154 podStartE2EDuration="1.705849154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:43:29.704262466 +0000 UTC m=+2844.959493884" watchObservedRunningTime="2026-07-17 20:43:29.705849154 +0000 UTC m=+2844.961080565" Jul 17 20:43:39.201249 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.201210 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:39.201677 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.201289 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:39.213920 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.213891 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:39.247837 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.247803 2578 scope.go:117] "RemoveContainer" containerID="f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078" Jul 17 20:43:39.743932 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.743890 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerStarted","Data":"d58fcfbd58f7ecd2da0b799046d012d56af7b084958d7a8343a0f03e902fd66d"} Jul 17 20:43:39.755172 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.755142 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:43:39.765191 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:39.765142 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" podStartSLOduration=16.082713749 podStartE2EDuration="18.765127973s" totalImagesPullingTime="2.682414224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:43:39.762993389 +0000 UTC m=+2855.018224805" watchObservedRunningTime="2026-07-17 20:43:39.765127973 +0000 UTC m=+2855.020359459" Jul 17 20:43:40.750742 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:40.750712 2578 generic.go:356] "Generic (PLEG): container finished" podID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerID="d58fcfbd58f7ecd2da0b799046d012d56af7b084958d7a8343a0f03e902fd66d" exitCode=6 Jul 17 20:43:40.751221 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:40.750785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerDied","Data":"d58fcfbd58f7ecd2da0b799046d012d56af7b084958d7a8343a0f03e902fd66d"} Jul 17 20:43:40.751221 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:40.750845 2578 scope.go:117] "RemoveContainer" containerID="f7cc8b5e68d726d563881972ad69be3dcd005c4c13b5b7b513330d4fc66d5078" Jul 17 20:43:40.751341 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:40.751240 2578 scope.go:117] "RemoveContainer" containerID="d58fcfbd58f7ecd2da0b799046d012d56af7b084958d7a8343a0f03e902fd66d" Jul 17 20:43:40.751468 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:43:40.751445 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-t8d2k_odh-ai-gateway-infra(500a86b5-d0f0-4e51-876d-289583edc9b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" Jul 17 20:43:40.780895 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:40.780863 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:41.888464 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:41.888435 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:42.008895 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.008801 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca\") pod \"500a86b5-d0f0-4e51-876d-289583edc9b2\" (UID: \"500a86b5-d0f0-4e51-876d-289583edc9b2\") " Jul 17 20:43:42.009129 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.009107 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "500a86b5-d0f0-4e51-876d-289583edc9b2" (UID: "500a86b5-d0f0-4e51-876d-289583edc9b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:43:42.109643 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.109578 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500a86b5-d0f0-4e51-876d-289583edc9b2-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:43:42.764514 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.764483 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" Jul 17 20:43:42.764751 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.764483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k" event={"ID":"500a86b5-d0f0-4e51-876d-289583edc9b2","Type":"ContainerDied","Data":"8c5f40b1c4cb3b7fe39ab8332780e70c8f2aa19c879928f0970f0aa64baf4ad1"} Jul 17 20:43:42.764751 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.764647 2578 scope.go:117] "RemoveContainer" containerID="d58fcfbd58f7ecd2da0b799046d012d56af7b084958d7a8343a0f03e902fd66d" Jul 17 20:43:42.788151 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.787984 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:42.790600 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:42.790578 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-t8d2k"] Jul 17 20:43:43.253770 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:43.253734 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" path="/var/lib/kubelet/pods/500a86b5-d0f0-4e51-876d-289583edc9b2/volumes" Jul 17 20:43:46.881819 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:43:46.881789 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:44:01.903072 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:01.902965 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" podUID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" containerName="main" containerID="cri-o://17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803" gracePeriod=45 Jul 17 20:44:02.161090 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.161027 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:44:02.304101 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304067 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304290 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304133 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304290 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304160 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304290 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304182 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktj7s\" (UniqueName: \"kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304290 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304200 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304290 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304284 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home\") pod \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\" (UID: \"99c1c8f1-f8f9-4454-b8a0-279722612fa2\") " Jul 17 20:44:02.304576 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache" (OuterVolumeSpecName: "model-cache") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:44:02.304576 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304447 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:44:02.304708 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304602 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.304708 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304602 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home" (OuterVolumeSpecName: "home") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:44:02.304708 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.304682 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.306428 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.306398 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:44:02.306428 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.306404 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm" (OuterVolumeSpecName: "dshm") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:44:02.306576 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.306493 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s" (OuterVolumeSpecName: "kube-api-access-ktj7s") pod "99c1c8f1-f8f9-4454-b8a0-279722612fa2" (UID: "99c1c8f1-f8f9-4454-b8a0-279722612fa2"). InnerVolumeSpecName "kube-api-access-ktj7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:44:02.406383 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.406339 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktj7s\" (UniqueName: \"kubernetes.io/projected/99c1c8f1-f8f9-4454-b8a0-279722612fa2-kube-api-access-ktj7s\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.406383 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.406383 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.406596 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.406398 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99c1c8f1-f8f9-4454-b8a0-279722612fa2-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.406596 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.406410 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1c8f1-f8f9-4454-b8a0-279722612fa2-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:44:02.888228 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.888196 2578 generic.go:356] "Generic (PLEG): container finished" podID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" containerID="17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803" exitCode=0 Jul 17 20:44:02.888409 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.888244 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" Jul 17 20:44:02.888409 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.888283 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" event={"ID":"99c1c8f1-f8f9-4454-b8a0-279722612fa2","Type":"ContainerDied","Data":"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803"} Jul 17 20:44:02.888409 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.888326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk" event={"ID":"99c1c8f1-f8f9-4454-b8a0-279722612fa2","Type":"ContainerDied","Data":"60f86ea263a7af605f7f051758f5a3edff182a80e57eb05f5de693b18abb9146"} Jul 17 20:44:02.888409 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.888344 2578 scope.go:117] "RemoveContainer" containerID="17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803" Jul 17 20:44:02.897711 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.897667 2578 scope.go:117] "RemoveContainer" containerID="17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803" Jul 17 20:44:02.897961 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:44:02.897944 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803\": container with ID starting with 17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803 not found: ID does not exist" containerID="17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803" Jul 17 20:44:02.898014 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.897971 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803"} err="failed to get container status \"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803\": rpc error: code = NotFound desc = could not find container \"17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803\": container with ID starting with 17106f7a069f01be75a9f9f27a32e48a0ab63b1e3d77dcd37fa5ad31f5e0c803 not found: ID does not exist" Jul 17 20:44:02.909948 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.909924 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:44:02.912127 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:02.912108 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5f0f4f/test-model-5f0f4f-kserve-54bff96f55-cdgdk"] Jul 17 20:44:03.254414 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:44:03.254382 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" path="/var/lib/kubelet/pods/99c1c8f1-f8f9-4454-b8a0-279722612fa2/volumes" Jul 17 20:45:00.133653 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.133619 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25"] Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.133994 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134005 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134013 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134019 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134026 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134031 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134047 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" containerName="main" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134052 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" containerName="main" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134118 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134125 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c1c8f1-f8f9-4454-b8a0-279722612fa2" containerName="main" Jul 17 20:45:00.134159 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.134133 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:45:00.137189 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.137172 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" Jul 17 20:45:00.139453 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.139428 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kc768\"" Jul 17 20:45:00.143973 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.143950 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25"] Jul 17 20:45:00.149594 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.149576 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" Jul 17 20:45:00.500088 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:00.498748 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25"] Jul 17 20:45:01.205751 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:01.205712 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" event={"ID":"fb529ad2-7f22-4d0f-ad03-525d211ff12d","Type":"ContainerStarted","Data":"f06f0235ba441215b18a64810e4fba3bf65ea574b58c60d5186c9d0e6e531790"} Jul 17 20:45:01.206162 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:01.205781 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" event={"ID":"fb529ad2-7f22-4d0f-ad03-525d211ff12d","Type":"ContainerStarted","Data":"c7a70d86e378ef0f5451aa5cbb486b1d7b80b3026e4e31b474d8cb3f070dfa48"} Jul 17 20:45:01.222591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:01.222533 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" podStartSLOduration=1.2225174380000001 podStartE2EDuration="1.222517438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:45:01.220293883 +0000 UTC m=+2936.475525314" watchObservedRunningTime="2026-07-17 20:45:01.222517438 +0000 UTC m=+2936.477748855" Jul 17 20:45:02.212830 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:02.212798 2578 generic.go:356] "Generic (PLEG): container finished" podID="fb529ad2-7f22-4d0f-ad03-525d211ff12d" containerID="f06f0235ba441215b18a64810e4fba3bf65ea574b58c60d5186c9d0e6e531790" exitCode=0 Jul 17 20:45:02.213210 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:02.212847 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" event={"ID":"fb529ad2-7f22-4d0f-ad03-525d211ff12d","Type":"ContainerDied","Data":"f06f0235ba441215b18a64810e4fba3bf65ea574b58c60d5186c9d0e6e531790"} Jul 17 20:45:03.353460 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:03.353435 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" Jul 17 20:45:04.225812 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:04.225784 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" Jul 17 20:45:04.225993 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:04.225822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25" event={"ID":"fb529ad2-7f22-4d0f-ad03-525d211ff12d","Type":"ContainerDied","Data":"c7a70d86e378ef0f5451aa5cbb486b1d7b80b3026e4e31b474d8cb3f070dfa48"} Jul 17 20:45:04.225993 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:04.225873 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7a70d86e378ef0f5451aa5cbb486b1d7b80b3026e4e31b474d8cb3f070dfa48" Jul 17 20:45:39.782098 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:39.782018 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25"] Jul 17 20:45:39.784167 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:39.784145 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-ad0e32-29738685-f4p25"] Jul 17 20:45:41.254032 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:45:41.254002 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb529ad2-7f22-4d0f-ad03-525d211ff12d" path="/var/lib/kubelet/pods/fb529ad2-7f22-4d0f-ad03-525d211ff12d/volumes" Jul 17 20:46:05.497062 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:05.497032 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:46:05.502859 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:05.502834 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:46:11.591760 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.591727 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:11.592234 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.592111 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb529ad2-7f22-4d0f-ad03-525d211ff12d" containerName="cleanup" Jul 17 20:46:11.592234 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.592122 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb529ad2-7f22-4d0f-ad03-525d211ff12d" containerName="cleanup" Jul 17 20:46:11.592234 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.592192 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="500a86b5-d0f0-4e51-876d-289583edc9b2" containerName="revoke-keys" Jul 17 20:46:11.592234 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.592198 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb529ad2-7f22-4d0f-ad03-525d211ff12d" containerName="cleanup" Jul 17 20:46:11.597155 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.597128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:11.601083 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.601059 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kc768\"" Jul 17 20:46:11.601333 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.601236 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 20:46:11.605813 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.605786 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:11.683866 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.683822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-8k5nl\" (UID: \"1a72d67f-ded1-4e63-ae28-72c164e60465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:11.784834 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.784783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-8k5nl\" (UID: \"1a72d67f-ded1-4e63-ae28-72c164e60465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:11.785388 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.785366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5420d171-8k5nl\" (UID: \"1a72d67f-ded1-4e63-ae28-72c164e60465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:11.911480 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:11.911445 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:12.045813 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:12.045789 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:12.047372 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:46:12.047347 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a72d67f_ded1_4e63_ae28_72c164e60465.slice/crio-6bf11e5510bae54f777de9ef22eb1e46c424cdc4d66dcc24914b94b69f28213e WatchSource:0}: Error finding container 6bf11e5510bae54f777de9ef22eb1e46c424cdc4d66dcc24914b94b69f28213e: Status 404 returned error can't find the container with id 6bf11e5510bae54f777de9ef22eb1e46c424cdc4d66dcc24914b94b69f28213e Jul 17 20:46:12.594127 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:12.594036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerStarted","Data":"989ba4d8cbfe5ba9c9283dd8251b544d9d92ea8926c54cd8afdf817f0e5e380a"} Jul 17 20:46:12.594127 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:12.594096 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerStarted","Data":"6bf11e5510bae54f777de9ef22eb1e46c424cdc4d66dcc24914b94b69f28213e"} Jul 17 20:46:12.613323 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:12.613272 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" podStartSLOduration=1.613254785 podStartE2EDuration="1.613254785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:46:12.611230762 +0000 UTC m=+3007.866462179" watchObservedRunningTime="2026-07-17 20:46:12.613254785 +0000 UTC m=+3007.868486201" Jul 17 20:46:13.601486 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:13.601451 2578 generic.go:356] "Generic (PLEG): container finished" podID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerID="989ba4d8cbfe5ba9c9283dd8251b544d9d92ea8926c54cd8afdf817f0e5e380a" exitCode=6 Jul 17 20:46:13.601892 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:13.601508 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerDied","Data":"989ba4d8cbfe5ba9c9283dd8251b544d9d92ea8926c54cd8afdf817f0e5e380a"} Jul 17 20:46:13.601892 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:13.601751 2578 scope.go:117] "RemoveContainer" containerID="989ba4d8cbfe5ba9c9283dd8251b544d9d92ea8926c54cd8afdf817f0e5e380a" Jul 17 20:46:14.608710 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:14.608595 2578 generic.go:356] "Generic (PLEG): container finished" podID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerID="626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6" exitCode=6 Jul 17 20:46:14.608710 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:14.608683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerDied","Data":"626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6"} Jul 17 20:46:14.609154 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:14.608746 2578 scope.go:117] "RemoveContainer" containerID="989ba4d8cbfe5ba9c9283dd8251b544d9d92ea8926c54cd8afdf817f0e5e380a" Jul 17 20:46:14.609154 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:14.608897 2578 scope.go:117] "RemoveContainer" containerID="626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6" Jul 17 20:46:14.609154 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:46:14.609129 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-8k5nl_odh-ai-gateway-infra(1a72d67f-ded1-4e63-ae28-72c164e60465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" Jul 17 20:46:15.616443 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:15.616419 2578 scope.go:117] "RemoveContainer" containerID="626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6" Jul 17 20:46:15.616869 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:46:15.616665 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-8k5nl_odh-ai-gateway-infra(1a72d67f-ded1-4e63-ae28-72c164e60465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" Jul 17 20:46:20.884513 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.884482 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:46:20.888081 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.888065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.891120 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.891097 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"openshift-service-ca.crt\"" Jul 17 20:46:20.891120 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.891112 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"auth-test-model-72eefe-kserve-self-signed-certs\"" Jul 17 20:46:20.891363 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.891111 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"default-dockercfg-lb468\"" Jul 17 20:46:20.891363 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.891136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"kube-root-ca.crt\"" Jul 17 20:46:20.896755 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.896733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:46:20.969412 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.969664 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.969664 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.969664 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969537 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzrmq\" (UniqueName: \"kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.969664 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:20.969898 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:20.969680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.070745 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.070707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.070745 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.070748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.070960 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.070880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzrmq\" (UniqueName: \"kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071001 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.070973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071045 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.070999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071045 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.071022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071361 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.071343 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071466 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.071376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.071466 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.071395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.073060 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.073038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.073311 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.073293 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.078112 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.078088 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzrmq\" (UniqueName: \"kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq\") pod \"auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.198640 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.198580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:21.334250 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.334222 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:46:21.335552 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:46:21.335518 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a52ccc3_26c3_4fe1_a31a_2dfcd270aa81.slice/crio-c8b46cb4b79164305b93978091c1631bf43b1b9ea0a4e1ef8060d5ac3c0c7556 WatchSource:0}: Error finding container c8b46cb4b79164305b93978091c1631bf43b1b9ea0a4e1ef8060d5ac3c0c7556: Status 404 returned error can't find the container with id c8b46cb4b79164305b93978091c1631bf43b1b9ea0a4e1ef8060d5ac3c0c7556 Jul 17 20:46:21.650391 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.650359 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" event={"ID":"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81","Type":"ContainerStarted","Data":"88d6aa716131182b0a45b8cd93dc1fa90306596ee976cfaddf78682533dea844"} Jul 17 20:46:21.650391 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.650403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" event={"ID":"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81","Type":"ContainerStarted","Data":"c8b46cb4b79164305b93978091c1631bf43b1b9ea0a4e1ef8060d5ac3c0c7556"} Jul 17 20:46:21.668907 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:21.668855 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" podStartSLOduration=1.6688408049999999 podStartE2EDuration="1.668840805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:46:21.666779977 +0000 UTC m=+3016.922011393" watchObservedRunningTime="2026-07-17 20:46:21.668840805 +0000 UTC m=+3016.924072222" Jul 17 20:46:30.248263 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:30.248231 2578 scope.go:117] "RemoveContainer" containerID="626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6" Jul 17 20:46:30.707591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:30.707555 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerStarted","Data":"ab5a478df9b7258f203332bf5af617bc517613d830624649da4c725ab010cdb0"} Jul 17 20:46:31.199245 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.199207 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:31.199432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.199265 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:31.212301 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.212276 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:31.715566 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.715539 2578 generic.go:356] "Generic (PLEG): container finished" podID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerID="ab5a478df9b7258f203332bf5af617bc517613d830624649da4c725ab010cdb0" exitCode=6 Jul 17 20:46:31.715992 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.715623 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerDied","Data":"ab5a478df9b7258f203332bf5af617bc517613d830624649da4c725ab010cdb0"} Jul 17 20:46:31.715992 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.715683 2578 scope.go:117] "RemoveContainer" containerID="626bf58f2126500d15c15a761257315383995360779a7078bb011d5070420de6" Jul 17 20:46:31.716123 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.716030 2578 scope.go:117] "RemoveContainer" containerID="ab5a478df9b7258f203332bf5af617bc517613d830624649da4c725ab010cdb0" Jul 17 20:46:31.716276 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:46:31.716253 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5420d171-8k5nl_odh-ai-gateway-infra(1a72d67f-ded1-4e63-ae28-72c164e60465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" Jul 17 20:46:31.727322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.727299 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:46:31.738915 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:31.738891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:32.863246 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:32.863219 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:32.879048 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:32.879017 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca\") pod \"1a72d67f-ded1-4e63-ae28-72c164e60465\" (UID: \"1a72d67f-ded1-4e63-ae28-72c164e60465\") " Jul 17 20:46:32.879538 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:32.879509 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a72d67f-ded1-4e63-ae28-72c164e60465" (UID: "1a72d67f-ded1-4e63-ae28-72c164e60465"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:46:32.979889 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:32.979795 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a72d67f-ded1-4e63-ae28-72c164e60465-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:46:33.728962 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:33.728929 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" Jul 17 20:46:33.728962 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:33.728950 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl" event={"ID":"1a72d67f-ded1-4e63-ae28-72c164e60465","Type":"ContainerDied","Data":"6bf11e5510bae54f777de9ef22eb1e46c424cdc4d66dcc24914b94b69f28213e"} Jul 17 20:46:33.729224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:33.729006 2578 scope.go:117] "RemoveContainer" containerID="ab5a478df9b7258f203332bf5af617bc517613d830624649da4c725ab010cdb0" Jul 17 20:46:33.744756 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:33.744729 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:33.751503 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:33.751480 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5420d171-8k5nl"] Jul 17 20:46:35.254292 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:35.254257 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" path="/var/lib/kubelet/pods/1a72d67f-ded1-4e63-ae28-72c164e60465/volumes" Jul 17 20:46:41.191481 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191445 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:46:41.191908 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191871 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.191908 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191883 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.191908 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191897 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.191908 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191903 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.192074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191966 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.192074 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.191975 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:46:41.196329 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.196312 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.203192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.202793 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"default-dockercfg-99jqs\"" Jul 17 20:46:41.203192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.202823 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"openshift-service-ca.crt\"" Jul 17 20:46:41.203192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.202848 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"kube-root-ca.crt\"" Jul 17 20:46:41.203192 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.203082 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"auth-test-model-327614-kserve-self-signed-certs\"" Jul 17 20:46:41.207246 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.207225 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:46:41.254274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.254477 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.254477 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv7cm\" (UniqueName: \"kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.254477 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.254628 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.254628 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.254557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355364 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355364 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv7cm\" (UniqueName: \"kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355635 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355884 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355936 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355900 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.355996 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.355975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.357801 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.357782 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.357980 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.357965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.363817 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.363796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv7cm\" (UniqueName: \"kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm\") pod \"auth-test-model-327614-kserve-55b74745f-xhzq2\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.506558 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.506419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:41.645824 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.645796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:46:41.646140 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:46:41.646112 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcce9680_bab8_4aee_b8ea_fd22f0399c3b.slice/crio-fbd09c94ce9fc6a6d177ade2c42ce5df94c360da699c874de2ecb18199f22c7d WatchSource:0}: Error finding container fbd09c94ce9fc6a6d177ade2c42ce5df94c360da699c874de2ecb18199f22c7d: Status 404 returned error can't find the container with id fbd09c94ce9fc6a6d177ade2c42ce5df94c360da699c874de2ecb18199f22c7d Jul 17 20:46:41.776424 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.776343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" event={"ID":"bcce9680-bab8-4aee-b8ea-fd22f0399c3b","Type":"ContainerStarted","Data":"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b"} Jul 17 20:46:41.776424 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.776382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" event={"ID":"bcce9680-bab8-4aee-b8ea-fd22f0399c3b","Type":"ContainerStarted","Data":"fbd09c94ce9fc6a6d177ade2c42ce5df94c360da699c874de2ecb18199f22c7d"} Jul 17 20:46:41.792720 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:41.792667 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" podStartSLOduration=0.792639622 podStartE2EDuration="792.639622ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:46:41.792533797 +0000 UTC m=+3037.047765421" watchObservedRunningTime="2026-07-17 20:46:41.792639622 +0000 UTC m=+3037.047871063" Jul 17 20:46:51.507631 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:51.507565 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:51.508035 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:51.507654 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:51.520195 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:51.520163 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:46:51.847204 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:46:51.847127 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:48:43.330796 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:43.330706 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:48:44.190542 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:44.190511 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:48:58.351497 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.351456 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" podUID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" containerName="main" containerID="cri-o://88d6aa716131182b0a45b8cd93dc1fa90306596ee976cfaddf78682533dea844" gracePeriod=45 Jul 17 20:48:58.544969 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.544939 2578 generic.go:356] "Generic (PLEG): container finished" podID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" containerID="88d6aa716131182b0a45b8cd93dc1fa90306596ee976cfaddf78682533dea844" exitCode=0 Jul 17 20:48:58.545120 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.545008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" event={"ID":"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81","Type":"ContainerDied","Data":"88d6aa716131182b0a45b8cd93dc1fa90306596ee976cfaddf78682533dea844"} Jul 17 20:48:58.604028 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.603962 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:48:58.679137 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679107 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679306 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679146 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679306 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679193 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzrmq\" (UniqueName: \"kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679306 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679211 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679306 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679282 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679472 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679322 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home\") pod \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\" (UID: \"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81\") " Jul 17 20:48:58.679530 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679441 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:58.679588 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679523 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:58.679719 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679694 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home" (OuterVolumeSpecName: "home") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:58.679776 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679720 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:58.679776 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.679734 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:58.681531 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.681501 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:48:58.681531 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.681516 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm" (OuterVolumeSpecName: "dshm") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:58.681695 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.681545 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq" (OuterVolumeSpecName: "kube-api-access-wzrmq") pod "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" (UID: "3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81"). InnerVolumeSpecName "kube-api-access-wzrmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:48:58.780930 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.780891 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:58.780930 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.780919 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:58.780930 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.780928 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:58.780930 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:58.780936 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzrmq\" (UniqueName: \"kubernetes.io/projected/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81-kube-api-access-wzrmq\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.211666 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.211624 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" podUID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" containerName="main" containerID="cri-o://59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b" gracePeriod=45 Jul 17 20:48:59.465947 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.465881 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:48:59.551637 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.551598 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" Jul 17 20:48:59.551637 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.551604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq" event={"ID":"3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81","Type":"ContainerDied","Data":"c8b46cb4b79164305b93978091c1631bf43b1b9ea0a4e1ef8060d5ac3c0c7556"} Jul 17 20:48:59.551866 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.551671 2578 scope.go:117] "RemoveContainer" containerID="88d6aa716131182b0a45b8cd93dc1fa90306596ee976cfaddf78682533dea844" Jul 17 20:48:59.553191 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.553164 2578 generic.go:356] "Generic (PLEG): container finished" podID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" containerID="59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b" exitCode=0 Jul 17 20:48:59.553322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.553194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" event={"ID":"bcce9680-bab8-4aee-b8ea-fd22f0399c3b","Type":"ContainerDied","Data":"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b"} Jul 17 20:48:59.553322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.553210 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" Jul 17 20:48:59.553322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.553227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2" event={"ID":"bcce9680-bab8-4aee-b8ea-fd22f0399c3b","Type":"ContainerDied","Data":"fbd09c94ce9fc6a6d177ade2c42ce5df94c360da699c874de2ecb18199f22c7d"} Jul 17 20:48:59.560731 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.560714 2578 scope.go:117] "RemoveContainer" containerID="59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b" Jul 17 20:48:59.567880 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.567856 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:48:59.569143 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.569110 2578 scope.go:117] "RemoveContainer" containerID="59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b" Jul 17 20:48:59.569625 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:48:59.569529 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b\": container with ID starting with 59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b not found: ID does not exist" containerID="59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b" Jul 17 20:48:59.569625 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.569570 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b"} err="failed to get container status \"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b\": rpc error: code = NotFound desc = could not find container \"59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b\": container with ID starting with 59530dd0e224f29d009fe34872b5d8fe28408fcf68dc95329924c4b09c579a1b not found: ID does not exist" Jul 17 20:48:59.571916 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.571897 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/auth-test-model-72eefe-kserve-6cbcf74f6c-rr2fq"] Jul 17 20:48:59.587832 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.587810 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.587931 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.587861 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.587931 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.587923 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.588009 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.587949 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.588009 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.587975 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv7cm\" (UniqueName: \"kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.588070 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588030 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache\") pod \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\" (UID: \"bcce9680-bab8-4aee-b8ea-fd22f0399c3b\") " Jul 17 20:48:59.588249 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588203 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home" (OuterVolumeSpecName: "home") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:59.588249 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588197 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:59.588435 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588332 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:59.588500 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588481 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.588554 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588506 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.588554 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.588520 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.590232 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.590209 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm" (OuterVolumeSpecName: "dshm") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:48:59.590232 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.590225 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm" (OuterVolumeSpecName: "kube-api-access-kv7cm") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "kube-api-access-kv7cm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:48:59.590358 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.590210 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bcce9680-bab8-4aee-b8ea-fd22f0399c3b" (UID: "bcce9680-bab8-4aee-b8ea-fd22f0399c3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:48:59.689755 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.689724 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.689755 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.689749 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.689755 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.689761 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv7cm\" (UniqueName: \"kubernetes.io/projected/bcce9680-bab8-4aee-b8ea-fd22f0399c3b-kube-api-access-kv7cm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:48:59.874657 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.874602 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:48:59.880062 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:48:59.880038 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-327614/auth-test-model-327614-kserve-55b74745f-xhzq2"] Jul 17 20:49:01.259931 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:01.257089 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" path="/var/lib/kubelet/pods/3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81/volumes" Jul 17 20:49:01.259931 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:01.257756 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" path="/var/lib/kubelet/pods/bcce9680-bab8-4aee-b8ea-fd22f0399c3b/volumes" Jul 17 20:49:14.668711 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.668673 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669285 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669303 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669331 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669341 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669368 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669379 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669473 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a52ccc3-26c3-4fe1-a31a-2dfcd270aa81" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669490 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcce9680-bab8-4aee-b8ea-fd22f0399c3b" containerName="main" Jul 17 20:49:14.671224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.669500 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a72d67f-ded1-4e63-ae28-72c164e60465" containerName="revoke-keys" Jul 17 20:49:14.673066 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.673048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.675411 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.675386 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"default-dockercfg-lb468\"" Jul 17 20:49:14.675526 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.675386 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"openshift-service-ca.crt\"" Jul 17 20:49:14.676401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.676377 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"sub-test-model-72eefe-kserve-self-signed-certs\"" Jul 17 20:49:14.676401 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.676401 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"kube-root-ca.crt\"" Jul 17 20:49:14.682382 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.682356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:49:14.841090 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.841090 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.841314 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.841314 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.841314 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.841314 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.841252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqqt\" (UniqueName: \"kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942285 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942285 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942285 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942574 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqqt\" (UniqueName: \"kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942574 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942366 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942574 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942774 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942735 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942833 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.942874 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.942844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.944453 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.944432 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.944849 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.944834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.950099 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.950079 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqqt\" (UniqueName: \"kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt\") pod \"sub-test-model-72eefe-kserve-d566f9b59-wndr9\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:14.985153 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:14.985123 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:15.119250 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:15.119217 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:49:15.120686 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:49:15.120655 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1630f156_7f19_4abe_9cb0_11a2bdb4c669.slice/crio-b645645ede19a1719b94c14a6ebde299c5b72d074ea0fa34b2374eacd1e56f79 WatchSource:0}: Error finding container b645645ede19a1719b94c14a6ebde299c5b72d074ea0fa34b2374eacd1e56f79: Status 404 returned error can't find the container with id b645645ede19a1719b94c14a6ebde299c5b72d074ea0fa34b2374eacd1e56f79 Jul 17 20:49:15.647738 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:15.647704 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" event={"ID":"1630f156-7f19-4abe-9cb0-11a2bdb4c669","Type":"ContainerStarted","Data":"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f"} Jul 17 20:49:15.647738 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:15.647744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" event={"ID":"1630f156-7f19-4abe-9cb0-11a2bdb4c669","Type":"ContainerStarted","Data":"b645645ede19a1719b94c14a6ebde299c5b72d074ea0fa34b2374eacd1e56f79"} Jul 17 20:49:15.665546 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:15.665492 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" podStartSLOduration=1.665473771 podStartE2EDuration="1.665473771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:49:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:49:15.6638176 +0000 UTC m=+3190.919049019" watchObservedRunningTime="2026-07-17 20:49:15.665473771 +0000 UTC m=+3190.920705187" Jul 17 20:49:24.985483 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:24.985441 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:24.985926 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:24.985504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:24.997777 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:24.997755 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:49:25.713043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:49:25.713014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:50:30.561341 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:30.561305 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:50:42.283334 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.283299 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:50:42.286831 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.286814 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.289680 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.289582 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-72eefe\"/\"rate-test-model-72eefe-kserve-self-signed-certs\"" Jul 17 20:50:42.295447 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.295420 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:50:42.373096 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.373297 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.373297 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq5vd\" (UniqueName: \"kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.373297 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373282 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.373432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.373432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.373342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.474816 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.474773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.474816 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.474815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475143 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.474929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475143 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.474966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475143 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.475043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475143 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.475064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq5vd\" (UniqueName: \"kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475335 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.475234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475373 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.475335 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.475408 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.475389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.477224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.477204 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.477482 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.477462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.483179 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.483154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq5vd\" (UniqueName: \"kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd\") pod \"rate-test-model-72eefe-kserve-867974b96c-j5njz\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.596734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.596640 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:42.730624 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:42.730583 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:50:42.732010 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:50:42.731984 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e9b637e_5dc0_4264_82ab_2ad0b4c7b89f.slice/crio-2c8e475a3288a2b1718132da5d9cc53e08c833f91815310efd29eae712320c4d WatchSource:0}: Error finding container 2c8e475a3288a2b1718132da5d9cc53e08c833f91815310efd29eae712320c4d: Status 404 returned error can't find the container with id 2c8e475a3288a2b1718132da5d9cc53e08c833f91815310efd29eae712320c4d Jul 17 20:50:43.130549 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:43.130511 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" event={"ID":"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f","Type":"ContainerStarted","Data":"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba"} Jul 17 20:50:43.130765 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:43.130564 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" event={"ID":"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f","Type":"ContainerStarted","Data":"2c8e475a3288a2b1718132da5d9cc53e08c833f91815310efd29eae712320c4d"} Jul 17 20:50:43.148774 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:43.148726 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" podStartSLOduration=1.148712027 podStartE2EDuration="1.148712027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:50:43.147406212 +0000 UTC m=+3278.402637630" watchObservedRunningTime="2026-07-17 20:50:43.148712027 +0000 UTC m=+3278.403943445" Jul 17 20:50:45.583224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.583149 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerName="main" containerID="cri-o://da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f" gracePeriod=45 Jul 17 20:50:45.702834 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.702796 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": dial tcp 10.132.0.59:8000: connect: connection refused" Jul 17 20:50:45.851248 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.851222 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:50:45.909783 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909742 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.909982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909811 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.909982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.909982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909922 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.909982 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909968 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.910215 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.909997 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjqqt\" (UniqueName: \"kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt\") pod \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\" (UID: \"1630f156-7f19-4abe-9cb0-11a2bdb4c669\") " Jul 17 20:50:45.910215 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910027 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:50:45.910215 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910138 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache" (OuterVolumeSpecName: "model-cache") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:50:45.910364 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910234 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home" (OuterVolumeSpecName: "home") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:50:45.910414 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910401 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:45.910464 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910421 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:45.910464 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.910435 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:45.912116 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.912091 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:50:45.912479 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.912459 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm" (OuterVolumeSpecName: "dshm") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:50:45.912547 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:45.912477 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt" (OuterVolumeSpecName: "kube-api-access-vjqqt") pod "1630f156-7f19-4abe-9cb0-11a2bdb4c669" (UID: "1630f156-7f19-4abe-9cb0-11a2bdb4c669"). InnerVolumeSpecName "kube-api-access-vjqqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:50:46.011320 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.011267 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1630f156-7f19-4abe-9cb0-11a2bdb4c669-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:46.011320 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.011313 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1630f156-7f19-4abe-9cb0-11a2bdb4c669-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:46.011320 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.011325 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjqqt\" (UniqueName: \"kubernetes.io/projected/1630f156-7f19-4abe-9cb0-11a2bdb4c669-kube-api-access-vjqqt\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:50:46.149268 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.149237 2578 generic.go:356] "Generic (PLEG): container finished" podID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerID="da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f" exitCode=0 Jul 17 20:50:46.149432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.149280 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" Jul 17 20:50:46.149432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.149323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" event={"ID":"1630f156-7f19-4abe-9cb0-11a2bdb4c669","Type":"ContainerDied","Data":"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f"} Jul 17 20:50:46.149432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.149371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9" event={"ID":"1630f156-7f19-4abe-9cb0-11a2bdb4c669","Type":"ContainerDied","Data":"b645645ede19a1719b94c14a6ebde299c5b72d074ea0fa34b2374eacd1e56f79"} Jul 17 20:50:46.149432 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.149393 2578 scope.go:117] "RemoveContainer" containerID="da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f" Jul 17 20:50:46.159493 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.159472 2578 scope.go:117] "RemoveContainer" containerID="da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f" Jul 17 20:50:46.159802 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:50:46.159775 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f\": container with ID starting with da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f not found: ID does not exist" containerID="da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f" Jul 17 20:50:46.159958 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.159803 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f"} err="failed to get container status \"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f\": rpc error: code = NotFound desc = could not find container \"da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f\": container with ID starting with da1f3b727129708d7fca9e8b45d654dbd0ef0d6df0e41cfa855037d236cf0d4f not found: ID does not exist" Jul 17 20:50:46.172034 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.172002 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:50:46.176451 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:46.176425 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/sub-test-model-72eefe-kserve-d566f9b59-wndr9"] Jul 17 20:50:47.253683 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:47.253655 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" path="/var/lib/kubelet/pods/1630f156-7f19-4abe-9cb0-11a2bdb4c669/volumes" Jul 17 20:50:52.597217 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:52.597179 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:52.597217 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:52.597228 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:52.609924 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:52.609897 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:50:53.200338 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:50:53.200313 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:51:05.533724 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:05.533689 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:51:05.542177 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:05.542157 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:51:05.646734 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:05.646703 2578 scope.go:117] "RemoveContainer" containerID="f06f0235ba441215b18a64810e4fba3bf65ea574b58c60d5186c9d0e6e531790" Jul 17 20:51:22.289649 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.289600 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:51:22.290134 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.289996 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerName="main" Jul 17 20:51:22.290134 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.290007 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerName="main" Jul 17 20:51:22.290134 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.290079 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1630f156-7f19-4abe-9cb0-11a2bdb4c669" containerName="main" Jul 17 20:51:22.294451 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.294422 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.298878 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.298856 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"default-dockercfg-99jqs\"" Jul 17 20:51:22.298878 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.298866 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"openshift-service-ca.crt\"" Jul 17 20:51:22.299070 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.298910 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"kube-root-ca.crt\"" Jul 17 20:51:22.299070 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.298866 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-327614\"/\"rate-test-model-327614-kserve-self-signed-certs\"" Jul 17 20:51:22.303927 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.303906 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:51:22.356485 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.356670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.356670 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8nv\" (UniqueName: \"kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.356783 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.356783 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.356896 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.356787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457758 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8nv\" (UniqueName: \"kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.457978 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.457885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.458286 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.458262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.458359 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.458294 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.458359 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.458327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.460026 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.460005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.460309 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.460290 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.465176 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.465151 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8nv\" (UniqueName: \"kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv\") pod \"rate-test-model-327614-kserve-7777f4669f-fhx5m\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.606320 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.606231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:22.741896 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:22.741872 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:51:22.743597 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:51:22.743568 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ec2a54b_8b43_461e_a20a_4e58bf75996e.slice/crio-dafe55ae046dd7f8e96030fc17beb8a98214d406b551ea6afb201aefa9ed777b WatchSource:0}: Error finding container dafe55ae046dd7f8e96030fc17beb8a98214d406b551ea6afb201aefa9ed777b: Status 404 returned error can't find the container with id dafe55ae046dd7f8e96030fc17beb8a98214d406b551ea6afb201aefa9ed777b Jul 17 20:51:23.363377 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:23.363343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" event={"ID":"8ec2a54b-8b43-461e-a20a-4e58bf75996e","Type":"ContainerStarted","Data":"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09"} Jul 17 20:51:23.363891 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:23.363391 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" event={"ID":"8ec2a54b-8b43-461e-a20a-4e58bf75996e","Type":"ContainerStarted","Data":"dafe55ae046dd7f8e96030fc17beb8a98214d406b551ea6afb201aefa9ed777b"} Jul 17 20:51:23.381824 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:23.381772 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" podStartSLOduration=1.38175456 podStartE2EDuration="1.38175456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:51:23.38035593 +0000 UTC m=+3318.635587348" watchObservedRunningTime="2026-07-17 20:51:23.38175456 +0000 UTC m=+3318.636986007" Jul 17 20:51:32.607014 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:32.606928 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:32.607014 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:32.606983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:32.620259 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:32.620228 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:51:33.431086 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:51:33.431055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:52:13.172381 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:13.172347 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:52:13.883535 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:13.883494 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:52:28.193478 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.193410 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" podUID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" containerName="main" containerID="cri-o://2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba" gracePeriod=45 Jul 17 20:52:28.453016 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.452950 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:52:28.595437 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595397 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595437 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq5vd\" (UniqueName: \"kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595473 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595500 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595548 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595709 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595586 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm\") pod \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\" (UID: \"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f\") " Jul 17 20:52:28.595926 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595763 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:28.595926 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595777 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:28.595926 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595877 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home" (OuterVolumeSpecName: "home") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:28.596059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595968 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.596059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595981 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.596059 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.595988 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.597644 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.597604 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm" (OuterVolumeSpecName: "dshm") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:28.597705 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.597654 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd" (OuterVolumeSpecName: "kube-api-access-vq5vd") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "kube-api-access-vq5vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:52:28.597744 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.597716 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" (UID: "4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:52:28.697442 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.697403 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq5vd\" (UniqueName: \"kubernetes.io/projected/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-kube-api-access-vq5vd\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.697442 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.697437 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.697442 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.697449 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:28.730114 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.730026 2578 generic.go:356] "Generic (PLEG): container finished" podID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" containerID="2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba" exitCode=0 Jul 17 20:52:28.730114 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.730065 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" Jul 17 20:52:28.730310 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.730108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" event={"ID":"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f","Type":"ContainerDied","Data":"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba"} Jul 17 20:52:28.730310 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.730150 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz" event={"ID":"4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f","Type":"ContainerDied","Data":"2c8e475a3288a2b1718132da5d9cc53e08c833f91815310efd29eae712320c4d"} Jul 17 20:52:28.730310 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.730168 2578 scope.go:117] "RemoveContainer" containerID="2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba" Jul 17 20:52:28.740043 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.740024 2578 scope.go:117] "RemoveContainer" containerID="2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba" Jul 17 20:52:28.740325 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:52:28.740307 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba\": container with ID starting with 2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba not found: ID does not exist" containerID="2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba" Jul 17 20:52:28.740379 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.740334 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba"} err="failed to get container status \"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba\": rpc error: code = NotFound desc = could not find container \"2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba\": container with ID starting with 2350ec296b42332a8d5d99a7bbd06433e405c3573319d25ed3e2f99ba0b485ba not found: ID does not exist" Jul 17 20:52:28.752433 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.752408 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:52:28.758487 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.758463 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-72eefe/rate-test-model-72eefe-kserve-867974b96c-j5njz"] Jul 17 20:52:28.903994 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:28.903954 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" podUID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" containerName="main" containerID="cri-o://ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09" gracePeriod=45 Jul 17 20:52:29.148186 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.148160 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:52:29.255496 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.255463 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" path="/var/lib/kubelet/pods/4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f/volumes" Jul 17 20:52:29.302716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302595 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.302716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302669 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.302716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302690 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.302716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302707 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.303054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.303054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302809 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js8nv\" (UniqueName: \"kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv\") pod \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\" (UID: \"8ec2a54b-8b43-461e-a20a-4e58bf75996e\") " Jul 17 20:52:29.303054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302910 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home" (OuterVolumeSpecName: "home") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:29.303054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.302930 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:29.303054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.303017 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:29.303280 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.303107 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.303280 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.303129 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.303280 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.303143 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.304920 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.304896 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm" (OuterVolumeSpecName: "dshm") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 20:52:29.305036 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.304992 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 20:52:29.305036 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.305020 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv" (OuterVolumeSpecName: "kube-api-access-js8nv") pod "8ec2a54b-8b43-461e-a20a-4e58bf75996e" (UID: "8ec2a54b-8b43-461e-a20a-4e58bf75996e"). InnerVolumeSpecName "kube-api-access-js8nv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 20:52:29.404360 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.404323 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec2a54b-8b43-461e-a20a-4e58bf75996e-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.404360 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.404357 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec2a54b-8b43-461e-a20a-4e58bf75996e-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.404360 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.404369 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js8nv\" (UniqueName: \"kubernetes.io/projected/8ec2a54b-8b43-461e-a20a-4e58bf75996e-kube-api-access-js8nv\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:29.737060 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.737026 2578 generic.go:356] "Generic (PLEG): container finished" podID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" containerID="ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09" exitCode=0 Jul 17 20:52:29.737238 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.737083 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" Jul 17 20:52:29.737238 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.737111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" event={"ID":"8ec2a54b-8b43-461e-a20a-4e58bf75996e","Type":"ContainerDied","Data":"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09"} Jul 17 20:52:29.737238 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.737166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m" event={"ID":"8ec2a54b-8b43-461e-a20a-4e58bf75996e","Type":"ContainerDied","Data":"dafe55ae046dd7f8e96030fc17beb8a98214d406b551ea6afb201aefa9ed777b"} Jul 17 20:52:29.737238 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.737191 2578 scope.go:117] "RemoveContainer" containerID="ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09" Jul 17 20:52:29.747054 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.747035 2578 scope.go:117] "RemoveContainer" containerID="ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09" Jul 17 20:52:29.747337 ip-10-0-142-80 kubenswrapper[2578]: E0717 20:52:29.747316 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09\": container with ID starting with ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09 not found: ID does not exist" containerID="ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09" Jul 17 20:52:29.747395 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.747346 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09"} err="failed to get container status \"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09\": rpc error: code = NotFound desc = could not find container \"ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09\": container with ID starting with ebd11d2fb18bf447fba368121e4ccc4ff47e842b66aa4098205481d5ca0e2d09 not found: ID does not exist" Jul 17 20:52:29.759562 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.759530 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:52:29.762766 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:29.762743 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-327614/rate-test-model-327614-kserve-7777f4669f-fhx5m"] Jul 17 20:52:31.259877 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:31.259843 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" path="/var/lib/kubelet/pods/8ec2a54b-8b43-461e-a20a-4e58bf75996e/volumes" Jul 17 20:52:33.079356 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079320 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9"] Jul 17 20:52:33.079775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079735 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" containerName="main" Jul 17 20:52:33.079775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079747 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" containerName="main" Jul 17 20:52:33.079775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079763 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" containerName="main" Jul 17 20:52:33.079775 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079769 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" containerName="main" Jul 17 20:52:33.079912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079828 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec2a54b-8b43-461e-a20a-4e58bf75996e" containerName="main" Jul 17 20:52:33.079912 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.079841 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9b637e-5dc0-4264-82ab-2ad0b4c7b89f" containerName="main" Jul 17 20:52:33.084066 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.084048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:33.086388 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.086359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kc768\"" Jul 17 20:52:33.086579 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.086372 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 20:52:33.090906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.090880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9"] Jul 17 20:52:33.241544 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.241512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9\" (UID: \"74ab6f7c-3c12-42f1-9552-3a451b0d5751\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:33.342133 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.342033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9\" (UID: \"74ab6f7c-3c12-42f1-9552-3a451b0d5751\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:33.342782 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.342758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9\" (UID: \"74ab6f7c-3c12-42f1-9552-3a451b0d5751\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:33.396547 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.396518 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:33.538245 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.538218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9"] Jul 17 20:52:33.539729 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:52:33.539699 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ab6f7c_3c12_42f1_9552_3a451b0d5751.slice/crio-ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d WatchSource:0}: Error finding container ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d: Status 404 returned error can't find the container with id ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d Jul 17 20:52:33.762439 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:33.762403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" event={"ID":"74ab6f7c-3c12-42f1-9552-3a451b0d5751","Type":"ContainerStarted","Data":"ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d"} Jul 17 20:52:34.769314 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:34.769283 2578 generic.go:356] "Generic (PLEG): container finished" podID="74ab6f7c-3c12-42f1-9552-3a451b0d5751" containerID="68fc2499a0dbb28dd182a19dd41ba18360c2577f3d5c90ba02550012c900699d" exitCode=0 Jul 17 20:52:34.769747 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:34.769330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" event={"ID":"74ab6f7c-3c12-42f1-9552-3a451b0d5751","Type":"ContainerDied","Data":"68fc2499a0dbb28dd182a19dd41ba18360c2577f3d5c90ba02550012c900699d"} Jul 17 20:52:35.917985 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:35.917959 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:35.962574 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:35.962537 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca\") pod \"74ab6f7c-3c12-42f1-9552-3a451b0d5751\" (UID: \"74ab6f7c-3c12-42f1-9552-3a451b0d5751\") " Jul 17 20:52:35.962915 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:35.962895 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74ab6f7c-3c12-42f1-9552-3a451b0d5751" (UID: "74ab6f7c-3c12-42f1-9552-3a451b0d5751"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 20:52:36.064136 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:36.064050 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ab6f7c-3c12-42f1-9552-3a451b0d5751-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\"" Jul 17 20:52:36.788367 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:36.788340 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" Jul 17 20:52:36.788528 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:36.788386 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9" event={"ID":"74ab6f7c-3c12-42f1-9552-3a451b0d5751","Type":"ContainerDied","Data":"ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d"} Jul 17 20:52:36.788528 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:52:36.788434 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebad89c6bd298fcc1dde148dcb5f60a01e15b4f06a14e524c567112dff4e670d" Jul 17 20:53:25.686060 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:25.686029 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664d5b5dd4-j2nf7_500a8d18-8b09-41f9-98e6-d7e376c3b820/authorino/0.log" Jul 17 20:53:31.697903 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:31.697880 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67bdc7ddc4-b97bd_a975e802-bcdd-479e-a286-4c47cef98646/copy-manifests/0.log" Jul 17 20:53:31.704535 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:31.704512 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67bdc7ddc4-b97bd_a975e802-bcdd-479e-a286-4c47cef98646/manager/0.log" Jul 17 20:53:33.393445 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:33.393418 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-72eefe-8p7s9_74ab6f7c-3c12-42f1-9552-3a451b0d5751/revoke-keys/0.log" Jul 17 20:53:34.365155 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.365132 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/util/0.log" Jul 17 20:53:34.372211 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.372191 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/pull/0.log" Jul 17 20:53:34.378708 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.378683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/extract/0.log" Jul 17 20:53:34.492446 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.492413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/util/0.log" Jul 17 20:53:34.499166 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.499140 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/pull/0.log" Jul 17 20:53:34.505708 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.505687 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/extract/0.log" Jul 17 20:53:34.614743 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.614715 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/util/0.log" Jul 17 20:53:34.620948 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.620849 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/pull/0.log" Jul 17 20:53:34.626917 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.626898 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/extract/0.log" Jul 17 20:53:34.737123 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.737096 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/util/0.log" Jul 17 20:53:34.743935 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.743913 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/pull/0.log" Jul 17 20:53:34.750067 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.750046 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/extract/0.log" Jul 17 20:53:34.866591 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:34.866554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664d5b5dd4-j2nf7_500a8d18-8b09-41f9-98e6-d7e376c3b820/authorino/0.log" Jul 17 20:53:35.191889 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:35.191859 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2tddb_6e0a67f4-aa55-4996-88b8-22acca436193/kuadrant-console-plugin/0.log" Jul 17 20:53:36.319391 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:36.319355 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fdbd865b7-hcxrm_85ccc331-03b5-437d-8bfc-ac910c5479ac/kube-auth-proxy/0.log" Jul 17 20:53:36.667570 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:36.667524 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q77xz_bb5acdaa-3ae6-4d88-8525-866c2de72469/payload-processing/0.log" Jul 17 20:53:36.777254 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:36.777224 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765ff78b4-jfc5r_a27a74c6-a582-4d59-b86c-b3b6e9e3e376/router/0.log" Jul 17 20:53:37.347144 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:37.347112 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gqr6m_8736cfab-fc84-4db8-9759-0269829b0d51/main/0.log" Jul 17 20:53:37.579224 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:37.579193 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sc5th_33853ccb-d364-4980-8158-0d3b10acf11d/main/0.log" Jul 17 20:53:37.691379 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:37.691351 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-bvqw7_ede1a23a-0c24-4af8-83bf-90b038876e63/main/0.log" Jul 17 20:53:43.370239 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.370196 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wlh9g/must-gather-9s7r4"] Jul 17 20:53:43.370785 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.370574 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ab6f7c-3c12-42f1-9552-3a451b0d5751" containerName="revoke-keys" Jul 17 20:53:43.370785 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.370585 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ab6f7c-3c12-42f1-9552-3a451b0d5751" containerName="revoke-keys" Jul 17 20:53:43.370785 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.370713 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74ab6f7c-3c12-42f1-9552-3a451b0d5751" containerName="revoke-keys" Jul 17 20:53:43.374228 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.374206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.376809 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.376787 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wlh9g\"/\"openshift-service-ca.crt\"" Jul 17 20:53:43.377051 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.377034 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wlh9g\"/\"default-dockercfg-kgm6k\"" Jul 17 20:53:43.377133 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.377062 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wlh9g\"/\"kube-root-ca.crt\"" Jul 17 20:53:43.386220 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.386193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wlh9g/must-gather-9s7r4"] Jul 17 20:53:43.499716 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.499680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8m6w\" (UniqueName: \"kubernetes.io/projected/622f6525-6fb8-4341-828f-8fb7f509ad82-kube-api-access-m8m6w\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.499906 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.499808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/622f6525-6fb8-4341-828f-8fb7f509ad82-must-gather-output\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.600328 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.600289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8m6w\" (UniqueName: \"kubernetes.io/projected/622f6525-6fb8-4341-828f-8fb7f509ad82-kube-api-access-m8m6w\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.600505 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.600411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/622f6525-6fb8-4341-828f-8fb7f509ad82-must-gather-output\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.600809 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.600793 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/622f6525-6fb8-4341-828f-8fb7f509ad82-must-gather-output\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.609396 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.609376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8m6w\" (UniqueName: \"kubernetes.io/projected/622f6525-6fb8-4341-828f-8fb7f509ad82-kube-api-access-m8m6w\") pod \"must-gather-9s7r4\" (UID: \"622f6525-6fb8-4341-828f-8fb7f509ad82\") " pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.684675 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.684645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" Jul 17 20:53:43.824007 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.823982 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wlh9g/must-gather-9s7r4"] Jul 17 20:53:43.825433 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:53:43.825401 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod622f6525_6fb8_4341_828f_8fb7f509ad82.slice/crio-154ab7611455d287c990f1e6d83bc1bf2274156da728bcad3d8d716bdf8d8a1d WatchSource:0}: Error finding container 154ab7611455d287c990f1e6d83bc1bf2274156da728bcad3d8d716bdf8d8a1d: Status 404 returned error can't find the container with id 154ab7611455d287c990f1e6d83bc1bf2274156da728bcad3d8d716bdf8d8a1d Jul 17 20:53:43.827422 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:43.827405 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 20:53:44.151600 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:44.151568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" event={"ID":"622f6525-6fb8-4341-828f-8fb7f509ad82","Type":"ContainerStarted","Data":"154ab7611455d287c990f1e6d83bc1bf2274156da728bcad3d8d716bdf8d8a1d"} Jul 17 20:53:45.159013 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:45.158967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" event={"ID":"622f6525-6fb8-4341-828f-8fb7f509ad82","Type":"ContainerStarted","Data":"ee6e3948b7a2e9a4dc2752cfc1337a17da35444b272122c3c9f48c9834d48408"} Jul 17 20:53:45.159511 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:45.159024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" event={"ID":"622f6525-6fb8-4341-828f-8fb7f509ad82","Type":"ContainerStarted","Data":"e5aca611b50498e59894e36a5e56f2cc3998670a6a4b85f8852c56768b0191dd"} Jul 17 20:53:45.176409 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:45.176351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wlh9g/must-gather-9s7r4" podStartSLOduration=1.341478733 podStartE2EDuration="2.176333033s" totalImagesPullingTime="834.8543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 20:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:53:45.175280501 +0000 UTC m=+3460.430511919" watchObservedRunningTime="2026-07-17 20:53:45.176333033 +0000 UTC m=+3460.431564450" Jul 17 20:53:46.357289 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:46.357250 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rqfrh_908d7010-dec8-4574-9ce4-29a5d5327bec/global-pull-secret-syncer/0.log" Jul 17 20:53:46.446828 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:46.446798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zfxzd_8fbb3889-3c38-4343-a98f-0b5326c1a9f4/konnectivity-agent/0.log" Jul 17 20:53:46.525322 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:46.525295 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-80.ec2.internal_ddc4a6bf9a8830c582e31a32617937e6/haproxy/0.log" Jul 17 20:53:50.360717 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.360685 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/extract/0.log" Jul 17 20:53:50.383453 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.383423 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/util/0.log" Jul 17 20:53:50.407548 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.407505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lcnwk_6055ab65-537a-41a1-87eb-9d9bae4fe02b/pull/0.log" Jul 17 20:53:50.442898 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.442843 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/extract/0.log" Jul 17 20:53:50.466683 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.466652 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/util/0.log" Jul 17 20:53:50.532115 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.532060 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tvxtd_208959b7-b157-4480-b570-4a7256ca1976/pull/0.log" Jul 17 20:53:50.563462 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.563426 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/extract/0.log" Jul 17 20:53:50.587543 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.587508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/util/0.log" Jul 17 20:53:50.610859 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.610779 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mq5gb_732066e2-6d11-4713-b407-9404558d7e9b/pull/0.log" Jul 17 20:53:50.640997 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.640971 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/extract/0.log" Jul 17 20:53:50.665826 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.665802 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/util/0.log" Jul 17 20:53:50.699669 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:50.699598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vz7h9_e2e1ef89-81e5-4031-8464-626ab305c3b5/pull/0.log" Jul 17 20:53:51.111999 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:51.111966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664d5b5dd4-j2nf7_500a8d18-8b09-41f9-98e6-d7e376c3b820/authorino/0.log" Jul 17 20:53:51.218004 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:51.217974 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2tddb_6e0a67f4-aa55-4996-88b8-22acca436193/kuadrant-console-plugin/0.log" Jul 17 20:53:53.029033 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.029010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/alertmanager/0.log" Jul 17 20:53:53.058078 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.058053 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/config-reloader/0.log" Jul 17 20:53:53.083017 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.082987 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/kube-rbac-proxy-web/0.log" Jul 17 20:53:53.112760 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.112731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/kube-rbac-proxy/0.log" Jul 17 20:53:53.141156 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.141129 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/kube-rbac-proxy-metric/0.log" Jul 17 20:53:53.166584 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.166548 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/prom-label-proxy/0.log" Jul 17 20:53:53.193869 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.193843 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_763ce8d8-936a-4a7d-8d43-d61373c27445/init-config-reloader/0.log" Jul 17 20:53:53.246528 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.246437 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xhr48_c014a045-d831-4d06-b875-f4460634be03/cluster-monitoring-operator/0.log" Jul 17 20:53:53.274771 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.274744 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-v7phx_ae49adf0-ffcb-46bf-828c-fcc66b70a542/kube-state-metrics/0.log" Jul 17 20:53:53.295109 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.295030 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-v7phx_ae49adf0-ffcb-46bf-828c-fcc66b70a542/kube-rbac-proxy-main/0.log" Jul 17 20:53:53.317486 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.317451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-v7phx_ae49adf0-ffcb-46bf-828c-fcc66b70a542/kube-rbac-proxy-self/0.log" Jul 17 20:53:53.373370 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.373297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-8x544_dda56631-feca-4d27-8fc5-63ddd1719c00/monitoring-plugin/0.log" Jul 17 20:53:53.488656 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.488628 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdmd_28c5127e-d725-412a-ac46-1597a907e696/node-exporter/0.log" Jul 17 20:53:53.511189 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.511142 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdmd_28c5127e-d725-412a-ac46-1597a907e696/kube-rbac-proxy/0.log" Jul 17 20:53:53.537575 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:53.537536 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdmd_28c5127e-d725-412a-ac46-1597a907e696/init-textfile/0.log" Jul 17 20:53:54.090209 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:54.090115 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-776b8bfc58-r8wqr_fe3b9b08-7538-4749-83d7-dd10c7c1295c/telemeter-client/0.log" Jul 17 20:53:54.111753 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:54.111721 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-776b8bfc58-r8wqr_fe3b9b08-7538-4749-83d7-dd10c7c1295c/reload/0.log" Jul 17 20:53:54.149166 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:54.149111 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-776b8bfc58-r8wqr_fe3b9b08-7538-4749-83d7-dd10c7c1295c/kube-rbac-proxy/0.log" Jul 17 20:53:55.556313 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:55.556282 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-97xw4_26b6965a-9faa-468e-9d98-0c63b8126a86/networking-console-plugin/0.log" Jul 17 20:53:56.299813 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:56.299764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/2.log" Jul 17 20:53:56.305077 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:56.305043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-7grmk_eba72d2c-0061-47b3-9339-2afcfebc950b/console-operator/3.log" Jul 17 20:53:57.281633 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.281537 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586"] Jul 17 20:53:57.289335 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.289297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.334462 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.334424 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586"] Jul 17 20:53:57.362549 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.362511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-lib-modules\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.362781 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.362702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-proc\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.362781 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.362747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gwjd\" (UniqueName: \"kubernetes.io/projected/802e6b66-7d5f-4406-8f55-84b1f83e5ece-kube-api-access-6gwjd\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.362907 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.362792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-sys\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.362961 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.362917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-podres\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464549 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-proc\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464758 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gwjd\" (UniqueName: \"kubernetes.io/projected/802e6b66-7d5f-4406-8f55-84b1f83e5ece-kube-api-access-6gwjd\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464758 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-sys\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464758 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-podres\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464758 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-proc\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464965 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-lib-modules\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464965 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-sys\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464965 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464826 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-podres\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.464965 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.464908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/802e6b66-7d5f-4406-8f55-84b1f83e5ece-lib-modules\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.476935 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.476868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gwjd\" (UniqueName: \"kubernetes.io/projected/802e6b66-7d5f-4406-8f55-84b1f83e5ece-kube-api-access-6gwjd\") pod \"perf-node-gather-daemonset-44586\" (UID: \"802e6b66-7d5f-4406-8f55-84b1f83e5ece\") " pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.602911 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.602826 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:57.768736 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.768688 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-6w47g_2975d364-f194-463a-8924-a1881774eb30/volume-data-source-validator/0.log" Jul 17 20:53:57.793001 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:57.792977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586"] Jul 17 20:53:57.794836 ip-10-0-142-80 kubenswrapper[2578]: W0717 20:53:57.794807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod802e6b66_7d5f_4406_8f55_84b1f83e5ece.slice/crio-31b2a9538dbc2275f4d56e57525cf934b9b6e6b7ef6bfa02f38475442ee335b7 WatchSource:0}: Error finding container 31b2a9538dbc2275f4d56e57525cf934b9b6e6b7ef6bfa02f38475442ee335b7: Status 404 returned error can't find the container with id 31b2a9538dbc2275f4d56e57525cf934b9b6e6b7ef6bfa02f38475442ee335b7 Jul 17 20:53:58.280006 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.279955 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" event={"ID":"802e6b66-7d5f-4406-8f55-84b1f83e5ece","Type":"ContainerStarted","Data":"68409613b3c9b13ab440a9692e2a761321b4c1c8942f043cbccb0129788cf42a"} Jul 17 20:53:58.280208 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.280042 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" event={"ID":"802e6b66-7d5f-4406-8f55-84b1f83e5ece","Type":"ContainerStarted","Data":"31b2a9538dbc2275f4d56e57525cf934b9b6e6b7ef6bfa02f38475442ee335b7"} Jul 17 20:53:58.280860 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.280834 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:53:58.299403 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.299343 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" podStartSLOduration=1.299322308 podStartE2EDuration="1.299322308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 20:53:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 20:53:58.298147475 +0000 UTC m=+3473.553378893" watchObservedRunningTime="2026-07-17 20:53:58.299322308 +0000 UTC m=+3473.554553726" Jul 17 20:53:58.936004 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.935979 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8pkg4_7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8/dns/0.log" Jul 17 20:53:58.971918 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:58.971886 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8pkg4_7ee5ac9d-d9b8-416a-a2d1-f47e3cc136c8/kube-rbac-proxy/0.log" Jul 17 20:53:59.106068 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:59.106040 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jftrg_67eb9374-9fb7-4b2f-8b35-eee4d7649140/dns-node-resolver/0.log" Jul 17 20:53:59.893712 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:53:59.893675 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cq8td_a0f3d694-caca-4f28-b36a-0d7e36afb234/node-ca/0.log" Jul 17 20:54:01.142813 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:01.142769 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fdbd865b7-hcxrm_85ccc331-03b5-437d-8bfc-ac910c5479ac/kube-auth-proxy/0.log" Jul 17 20:54:01.296366 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:01.296326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q77xz_bb5acdaa-3ae6-4d88-8525-866c2de72469/payload-processing/0.log" Jul 17 20:54:01.323483 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:01.323455 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765ff78b4-jfc5r_a27a74c6-a582-4d59-b86c-b3b6e9e3e376/router/0.log" Jul 17 20:54:02.268566 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.268519 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zsckk_34abb3e7-a612-4773-9220-b47320b4ce85/serve-healthcheck-canary/0.log" Jul 17 20:54:02.805623 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.805580 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-2v946_20a98849-040d-4d02-8485-70ed0281cf4f/insights-operator/0.log" Jul 17 20:54:02.808634 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.808592 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-2v946_20a98849-040d-4d02-8485-70ed0281cf4f/insights-operator/1.log" Jul 17 20:54:02.831428 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.831402 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wrtl_7ff80eb7-0b29-44a8-ab00-501321c6542e/kube-rbac-proxy/0.log" Jul 17 20:54:02.855861 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.855831 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wrtl_7ff80eb7-0b29-44a8-ab00-501321c6542e/exporter/0.log" Jul 17 20:54:02.880274 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:02.880237 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wrtl_7ff80eb7-0b29-44a8-ab00-501321c6542e/extractor/0.log" Jul 17 20:54:05.310546 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:05.310516 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wlh9g/perf-node-gather-daemonset-44586" Jul 17 20:54:05.763866 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:05.763833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67bdc7ddc4-b97bd_a975e802-bcdd-479e-a286-4c47cef98646/manager/0.log" Jul 17 20:54:05.788765 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:05.788739 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67bdc7ddc4-b97bd_a975e802-bcdd-479e-a286-4c47cef98646/copy-manifests/0.log" Jul 17 20:54:07.063002 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:07.062970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c54c94474-b79zn_f93ca144-8090-4bfc-bb10-f31923b91078/manager/0.log" Jul 17 20:54:07.391256 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:07.391209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fdbd865b7-hcxrm_85ccc331-03b5-437d-8bfc-ac910c5479ac/kube-auth-proxy/0.log" Jul 17 20:54:07.488987 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:07.488899 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q77xz_bb5acdaa-3ae6-4d88-8525-866c2de72469/payload-processing/0.log" Jul 17 20:54:07.507103 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:07.507069 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765ff78b4-jfc5r_a27a74c6-a582-4d59-b86c-b3b6e9e3e376/router/0.log" Jul 17 20:54:08.970453 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:08.970415 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fdbd865b7-hcxrm_85ccc331-03b5-437d-8bfc-ac910c5479ac/kube-auth-proxy/0.log" Jul 17 20:54:09.076910 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:09.076873 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q77xz_bb5acdaa-3ae6-4d88-8525-866c2de72469/payload-processing/0.log" Jul 17 20:54:09.095764 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:09.095739 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765ff78b4-jfc5r_a27a74c6-a582-4d59-b86c-b3b6e9e3e376/router/0.log" Jul 17 20:54:12.783738 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:12.783704 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-g6c7k_1be32d05-bf30-4d24-9ce0-483471ef371c/kube-storage-version-migrator-operator/1.log" Jul 17 20:54:12.784598 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:12.784578 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-g6c7k_1be32d05-bf30-4d24-9ce0-483471ef371c/kube-storage-version-migrator-operator/0.log" Jul 17 20:54:13.837018 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.836989 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-87pgj_002b2b02-4cf2-49f7-8ab6-cac22026c906/kube-multus/0.log" Jul 17 20:54:13.862477 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.862451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/kube-multus-additional-cni-plugins/0.log" Jul 17 20:54:13.888851 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.888819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/egress-router-binary-copy/0.log" Jul 17 20:54:13.913666 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.913604 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/cni-plugins/0.log" Jul 17 20:54:13.940984 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.940960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/bond-cni-plugin/0.log" Jul 17 20:54:13.963461 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.963435 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/routeoverride-cni/0.log" Jul 17 20:54:13.990539 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:13.990513 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/whereabouts-cni-bincopy/0.log" Jul 17 20:54:14.013112 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:14.013083 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8sz_a69bc2e5-8958-461f-a7e9-dca1c0c070c5/whereabouts-cni/0.log" Jul 17 20:54:14.492443 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:14.492413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8gscn_1d6ecb91-80ae-4221-9797-50fa1be7eda1/network-metrics-daemon/0.log" Jul 17 20:54:14.513180 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:14.513150 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8gscn_1d6ecb91-80ae-4221-9797-50fa1be7eda1/kube-rbac-proxy/0.log" Jul 17 20:54:15.459413 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.459380 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/ovn-controller/0.log" Jul 17 20:54:15.481523 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.481494 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/ovn-acl-logging/0.log" Jul 17 20:54:15.506362 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.506336 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/kube-rbac-proxy-node/0.log" Jul 17 20:54:15.529605 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.529574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/kube-rbac-proxy-ovn-metrics/0.log" Jul 17 20:54:15.548711 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.548638 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/northd/0.log" Jul 17 20:54:15.569760 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.569714 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/nbdb/0.log" Jul 17 20:54:15.591904 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.591875 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/sbdb/0.log" Jul 17 20:54:15.671378 ip-10-0-142-80 kubenswrapper[2578]: I0717 20:54:15.671323 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dzmx_47f1e417-214a-49d4-aca6-7b0bb1fc0627/ovnkube-controller/0.log"