Jul 08 15:50:54.618833 ip-10-0-136-198 systemd[1]: Starting Kubernetes Kubelet... Jul 08 15:50:55.025613 ip-10-0-136-198 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 15:50:55.025613 ip-10-0-136-198 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 15:50:55.025613 ip-10-0-136-198 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 15:50:55.025613 ip-10-0-136-198 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 15:50:55.025613 ip-10-0-136-198 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 15:50:55.026595 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.026453 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 15:50:55.029974 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029959 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.029974 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029973 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029977 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029981 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029984 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029988 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029991 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029994 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.029997 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030000 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030003 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030005 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030013 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030016 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030018 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030021 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030024 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030026 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030029 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030032 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030034 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.030036 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030037 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030039 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030043 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030047 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030052 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030055 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030058 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030061 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030064 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030066 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030069 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030071 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030074 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030076 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030079 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030081 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030083 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030086 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030088 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.030508 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030090 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030093 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030095 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030099 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030102 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030111 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030113 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030116 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030118 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030121 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030123 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030125 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030128 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030130 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030134 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030137 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030140 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030142 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030145 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.030979 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030147 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030150 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030153 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030155 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030158 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030160 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030163 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030165 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030167 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030171 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030174 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030176 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030178 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030181 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030183 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030186 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030189 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030192 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030194 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030197 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.031591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030200 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030202 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030205 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030207 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030209 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030212 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030215 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030217 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030220 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030223 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030225 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030228 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030232 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030235 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030238 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030240 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030243 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030246 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030248 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.032074 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030250 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030924 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030931 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030934 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030937 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030940 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030943 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030946 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030949 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030951 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030954 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030957 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030959 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030962 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030965 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030968 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030970 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030973 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030975 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030978 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.032529 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030981 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030984 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030987 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030989 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030992 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030994 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030997 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.030999 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031002 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031005 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031007 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031010 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031012 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031015 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031017 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031019 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031022 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031026 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031029 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.033034 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031032 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031035 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031038 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031040 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031043 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031045 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031048 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031051 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031054 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031056 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031059 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031061 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031063 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031066 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031069 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031073 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031077 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031080 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031083 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.033494 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031085 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031088 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031090 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031093 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031096 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031099 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031103 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031106 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031109 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031112 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031115 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031117 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031120 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031122 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031125 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031128 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031130 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031133 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031135 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.033972 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031139 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031141 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031145 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031147 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031150 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031152 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031155 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031158 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031160 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031162 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031165 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031167 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031170 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031172 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031175 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031177 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031179 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031182 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031184 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031187 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.034438 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031189 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031192 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031194 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031529 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031533 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031545 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031548 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031551 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031554 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031557 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031559 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031562 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031581 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031584 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031588 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031590 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031593 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031596 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031598 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031601 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.034931 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031604 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031607 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031609 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031612 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031614 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031617 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031619 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031622 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031625 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031628 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031630 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031633 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031636 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031638 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031641 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031643 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031646 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031649 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031651 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.035427 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031653 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031656 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031658 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031661 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031664 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031667 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031669 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031672 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031676 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031678 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031680 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031683 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031685 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031688 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031690 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031692 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031695 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031697 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031700 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031702 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.035924 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031704 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031706 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031709 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031711 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031714 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031716 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031719 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031723 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031726 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031728 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031731 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031733 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031736 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031738 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031741 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031743 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031746 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031748 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031751 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.036462 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031755 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031758 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031762 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031764 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031767 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031770 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031772 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031775 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031777 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031780 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031782 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031785 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031789 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031792 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031795 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031798 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031801 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031803 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031805 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031808 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.036920 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031812 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031815 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031822 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.031824 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031889 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031900 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031905 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031910 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031914 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031917 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031921 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031925 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031928 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031931 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031934 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031938 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031941 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031943 2566 flags.go:64] FLAG: --cgroup-root="" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031946 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031949 2566 flags.go:64] FLAG: --client-ca-file="" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031951 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031954 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031958 2566 flags.go:64] FLAG: --cluster-domain="" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031961 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 15:50:55.037399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031964 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031966 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031970 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031973 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031976 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031979 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031982 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031985 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031989 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031992 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031997 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.031999 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032003 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032006 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032009 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032011 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032014 2566 flags.go:64] FLAG: --enable-server="true" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032017 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032021 2566 flags.go:64] FLAG: --event-burst="100" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032024 2566 flags.go:64] FLAG: --event-qps="50" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032027 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032030 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032033 2566 flags.go:64] FLAG: --eviction-hard="" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032036 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 15:50:55.038048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032045 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032048 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032051 2566 flags.go:64] FLAG: --eviction-soft="" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032054 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032056 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032059 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032062 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032064 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032067 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032070 2566 flags.go:64] FLAG: --feature-gates="" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032077 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032080 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032083 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032086 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032089 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032092 2566 flags.go:64] FLAG: --help="false" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032096 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032099 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032103 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032106 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032109 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032112 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032115 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 15:50:55.038636 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032117 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032120 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032123 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032125 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032128 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032131 2566 flags.go:64] FLAG: --kube-reserved="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032134 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032136 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032139 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032142 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032145 2566 flags.go:64] FLAG: --lock-file="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032147 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032150 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032153 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032158 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032160 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032163 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032165 2566 flags.go:64] FLAG: --logging-format="text" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032168 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032171 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032174 2566 flags.go:64] FLAG: --manifest-url="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032176 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032180 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032183 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032187 2566 flags.go:64] FLAG: --max-pods="110" Jul 08 15:50:55.039436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032191 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032194 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032198 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032201 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032204 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032206 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032209 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032215 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032218 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032221 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032224 2566 flags.go:64] FLAG: --pod-cidr="" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032227 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032232 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032235 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032238 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032240 2566 flags.go:64] FLAG: --port="10250" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032243 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032246 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00ae5bd92173f8682" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032249 2566 flags.go:64] FLAG: --qos-reserved="" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032252 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032255 2566 flags.go:64] FLAG: --register-node="true" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032258 2566 flags.go:64] FLAG: --register-with-taints="" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032261 2566 flags.go:64] FLAG: --registry-burst="10" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032263 2566 flags.go:64] FLAG: --registry-qps="5" Jul 08 15:50:55.040191 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032266 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032269 2566 flags.go:64] FLAG: --reserved-memory="" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032272 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032275 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032278 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032280 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032283 2566 flags.go:64] FLAG: --runonce="false" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032285 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032288 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032292 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032296 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032299 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032301 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032304 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032307 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032310 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032313 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032315 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032318 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032321 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032323 2566 flags.go:64] FLAG: --system-cgroups="" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032326 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032331 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032334 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032336 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 15:50:55.040905 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032340 2566 flags.go:64] FLAG: --tls-min-version="" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032343 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032346 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032349 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032351 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032354 2566 flags.go:64] FLAG: --v="2" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032357 2566 flags.go:64] FLAG: --version="false" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032361 2566 flags.go:64] FLAG: --vmodule="" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032364 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.032367 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033246 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033252 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033255 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033258 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033261 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033263 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033267 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033271 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033273 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033276 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033279 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033281 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.041965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033284 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033286 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033289 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033292 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033294 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033297 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033300 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033302 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033305 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033307 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033309 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033312 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033314 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033317 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033320 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033323 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033325 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033327 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033330 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033332 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.042808 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033334 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033337 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033339 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033341 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033344 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033346 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033349 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033352 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033355 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033358 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033362 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033365 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033367 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033370 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033372 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033374 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033377 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033379 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033382 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.043285 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033384 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033387 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033389 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033392 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033394 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033396 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033399 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033401 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033404 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033407 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033409 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033411 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033414 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033416 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033420 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033423 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033427 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033429 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033432 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.043789 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033435 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033438 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033440 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033443 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033445 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033447 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033451 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033454 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033457 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033460 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033462 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033465 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033467 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033470 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033472 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033474 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033477 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033479 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033481 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.044238 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033484 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033486 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033489 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033491 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033493 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033496 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033498 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033501 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033504 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.033507 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.034075 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.043294 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.043307 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.044146 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.044154 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 15:50:55.044722 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044209 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044214 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044217 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044221 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044224 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044228 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044231 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044233 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044236 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044239 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044242 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044244 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044247 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044250 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044252 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044255 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044258 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044260 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044264 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.045090 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044266 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044269 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044272 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044275 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044277 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044280 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044283 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044285 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044288 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044291 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044293 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044296 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044298 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044301 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044304 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044307 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044309 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044312 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044314 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044317 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.045544 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044319 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044322 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044325 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044327 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044329 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044332 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044334 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044337 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044340 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044343 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044345 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044347 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044350 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044352 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044355 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044357 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044360 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044362 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044364 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044367 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.046041 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044369 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044372 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044374 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044377 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044380 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044384 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044387 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044390 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044392 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044395 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044398 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044400 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044403 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044406 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044410 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044413 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044415 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044419 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044423 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.046553 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044425 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044429 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044432 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044435 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044438 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044441 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044443 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044446 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044448 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044451 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044453 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044456 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044458 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044461 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044463 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044466 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044469 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044471 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044474 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044476 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.047048 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044479 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.044484 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044610 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044616 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044619 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044622 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044625 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044627 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044630 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044632 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044635 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044637 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044640 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044643 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044646 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 15:50:55.047515 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044648 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044651 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044654 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044656 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044658 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044661 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044664 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044667 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044669 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044672 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044674 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044677 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044680 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044683 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044685 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044688 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044690 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044693 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044695 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044698 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 15:50:55.048018 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044700 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044703 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044705 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044708 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044710 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044712 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044715 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044717 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044720 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044722 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044724 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044727 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044731 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044733 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044736 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044739 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044741 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044744 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044746 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044749 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 15:50:55.048504 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044751 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044754 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044756 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044759 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044761 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044763 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044766 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044768 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044771 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044773 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044776 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044778 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044780 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044783 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044785 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044788 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044790 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044792 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044795 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044797 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 15:50:55.049006 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044800 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044802 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044804 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044807 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044810 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044813 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044815 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044818 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044820 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044823 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044825 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044828 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044830 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044834 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044837 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044840 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044842 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044845 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044848 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 15:50:55.049483 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044852 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044855 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044859 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044862 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044865 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044868 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:55.044870 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.044875 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.044970 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.046875 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.048274 2566 server.go:1019] "Starting client certificate rotation" Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.048370 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 15:50:55.049995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.048401 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 15:50:55.050738 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.050725 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 15:50:55.053459 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.053446 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 15:50:55.064034 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.064017 2566 log.go:25] "Validated CRI v1 runtime API" Jul 08 15:50:55.066490 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.066473 2566 log.go:25] "Validated CRI v1 image API" Jul 08 15:50:55.067299 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.067283 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 15:50:55.072489 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.072470 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 890bbae2-7324-4573-83ae-aacf4c359681:/dev/nvme0n1p4 daf9c6b2-9ffe-4f53-8bd7-f6ceb5bfb57e:/dev/nvme0n1p3] Jul 08 15:50:55.072563 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.072489 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 15:50:55.077684 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.077667 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 15:50:55.078247 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.078149 2566 manager.go:216] Machine: {Timestamp:2026-07-08 15:50:55.076959517 +0000 UTC m=+0.339978178 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099280 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26de0f8a9c82e0e42de132992209b8 SystemUUID:ec26de0f-8a9c-82e0-e42d-e132992209b8 BootID:8831de15-8b6d-43f4-b1d7-5228b986fd0c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:74:e1:04:44:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:74:e1:04:44:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:f8:9b:68:cc:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 15:50:55.078247 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.078244 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 15:50:55.078355 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.078339 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 15:50:55.080523 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.080481 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 15:50:55.080897 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.080531 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-198.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 15:50:55.081407 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.081396 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 15:50:55.081441 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.081410 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 15:50:55.081441 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.081418 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 15:50:55.082173 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.082163 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 15:50:55.083546 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.083528 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 15:50:55.086804 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086793 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 08 15:50:55.086913 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086903 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 15:50:55.086954 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086932 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 08 15:50:55.086954 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086940 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 15:50:55.086954 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086950 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 15:50:55.087068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086959 2566 kubelet.go:398] "Adding apiserver pod source" Jul 08 15:50:55.087068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.086967 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 15:50:55.087309 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.087297 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 15:50:55.087344 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.087318 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 15:50:55.089344 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.089328 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 15:50:55.091185 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.091171 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 15:50:55.091251 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.091200 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 15:50:55.092164 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092151 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 15:50:55.092207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092170 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 15:50:55.092207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092177 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 15:50:55.092207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092186 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 15:50:55.092207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092199 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 15:50:55.092207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092206 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092214 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092219 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092225 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092230 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092238 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 15:50:55.092339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.092244 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 15:50:55.093934 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.093923 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 15:50:55.093934 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.093933 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 15:50:55.096854 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.096840 2566 server.go:1268] "Started kubelet" Jul 08 15:50:55.096984 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.096968 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-198.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 15:50:55.097063 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.097031 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 15:50:55.097098 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.097053 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 15:50:55.097126 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.097109 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 15:50:55.097621 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.097595 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 15:50:55.097635 ip-10-0-136-198 systemd[1]: Started Kubernetes Kubelet. Jul 08 15:50:55.098327 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.097592 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 15:50:55.098804 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.098789 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 15:50:55.099396 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.099384 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 08 15:50:55.103513 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.102532 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-198.ec2.internal.18c05b50d0c53776 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-198.ec2.internal,UID:ip-10-0-136-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-198.ec2.internal,},FirstTimestamp:2026-07-08 15:50:55.09681343 +0000 UTC m=+0.359832090,LastTimestamp:2026-07-08 15:50:55.09681343 +0000 UTC m=+0.359832090,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-198.ec2.internal,}" Jul 08 15:50:55.103622 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.103526 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 15:50:55.104293 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104279 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 15:50:55.104361 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104295 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 15:50:55.104408 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.104395 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.104834 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104817 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 15:50:55.104834 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104833 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 15:50:55.104962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104880 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 15:50:55.104962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104943 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 08 15:50:55.104962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.104953 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 08 15:50:55.105362 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105346 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 15:50:55.105362 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105364 2566 factory.go:55] Registering systemd factory Jul 08 15:50:55.105477 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105372 2566 factory.go:223] Registration of the systemd container factory successfully Jul 08 15:50:55.105617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105600 2566 factory.go:153] Registering CRI-O factory Jul 08 15:50:55.105617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105616 2566 factory.go:223] Registration of the crio container factory successfully Jul 08 15:50:55.105766 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105632 2566 factory.go:103] Registering Raw factory Jul 08 15:50:55.105766 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.105642 2566 manager.go:1196] Started watching for new ooms in manager Jul 08 15:50:55.106329 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.106052 2566 manager.go:349] Starting recovery of all containers Jul 08 15:50:55.114652 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.114623 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 15:50:55.114652 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.114626 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 15:50:55.114960 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.114934 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 15:50:55.118012 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.117996 2566 manager.go:354] Recovery completed Jul 08 15:50:55.122140 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.122128 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.123891 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.123862 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.123974 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.123901 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.123974 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.123911 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.124307 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124295 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 15:50:55.124307 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124304 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.124379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124306 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 15:50:55.124379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124356 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 08 15:50:55.124978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124965 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.125023 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.124992 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.125023 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.125002 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.125439 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.125428 2566 policy_none.go:49] "None policy: Start" Jul 08 15:50:55.125473 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.125443 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 15:50:55.125473 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.125453 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 15:50:55.126297 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.126238 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-198.ec2.internal.18c05b50d26255ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-198.ec2.internal,UID:ip-10-0-136-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-198.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-198.ec2.internal,},FirstTimestamp:2026-07-08 15:50:55.123887534 +0000 UTC m=+0.386906197,LastTimestamp:2026-07-08 15:50:55.123887534 +0000 UTC m=+0.386906197,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-198.ec2.internal,}" Jul 08 15:50:55.126591 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.126562 2566 policy_none.go:47] "Start" Jul 08 15:50:55.136350 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.136203 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-198.ec2.internal.18c05b50d2629e0b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-198.ec2.internal,UID:ip-10-0-136-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-198.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-198.ec2.internal,},FirstTimestamp:2026-07-08 15:50:55.123906059 +0000 UTC m=+0.386924722,LastTimestamp:2026-07-08 15:50:55.123906059 +0000 UTC m=+0.386924722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-198.ec2.internal,}" Jul 08 15:50:55.145427 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.145403 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bmfpm" Jul 08 15:50:55.146872 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.146796 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-198.ec2.internal.18c05b50d262bfe7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-198.ec2.internal,UID:ip-10-0-136-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-198.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-198.ec2.internal,},FirstTimestamp:2026-07-08 15:50:55.123914727 +0000 UTC m=+0.386933390,LastTimestamp:2026-07-08 15:50:55.123914727 +0000 UTC m=+0.386933390,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-198.ec2.internal,}" Jul 08 15:50:55.153604 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.153583 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bmfpm" Jul 08 15:50:55.156331 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.156268 2566 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-198.ec2.internal.18c05b50d26255ae\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-198.ec2.internal.18c05b50d26255ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-198.ec2.internal,UID:ip-10-0-136-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-198.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-198.ec2.internal,},FirstTimestamp:2026-07-08 15:50:55.123887534 +0000 UTC m=+0.386906197,LastTimestamp:2026-07-08 15:50:55.124978806 +0000 UTC m=+0.387997466,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-198.ec2.internal,}" Jul 08 15:50:55.158096 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.158083 2566 manager.go:337] "Starting Device Plugin manager" Jul 08 15:50:55.158160 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.158110 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 15:50:55.158160 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.158120 2566 server.go:87] "Starting device plugin registration server" Jul 08 15:50:55.158287 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.158277 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 15:50:55.158316 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.158290 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 15:50:55.159255 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.159107 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 15:50:55.159255 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.159178 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 15:50:55.159255 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.159187 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 15:50:55.160135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.160116 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 15:50:55.160206 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.160166 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.258384 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.258359 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.259361 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.259346 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.259421 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.259378 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.259421 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.259390 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.259421 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.259412 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.262532 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.262509 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 15:50:55.262633 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.262549 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 15:50:55.262633 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.262577 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 15:50:55.262633 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.262613 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 15:50:55.264458 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.264443 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 15:50:55.268985 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.268972 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.269051 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.268991 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-198.ec2.internal\": node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.314702 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.314655 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.363055 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.363035 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal"] Jul 08 15:50:55.363108 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.363102 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.364532 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.364518 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.364623 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.364546 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.364623 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.364561 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.365786 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.365774 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.365906 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.365894 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.365943 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.365920 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.366455 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366440 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.366455 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366451 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.366548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366468 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.366548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366474 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.366548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366486 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.366548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.366503 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.367742 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.367721 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.367742 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.367744 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 15:50:55.368336 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.368322 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientMemory" Jul 08 15:50:55.368405 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.368347 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 15:50:55.368405 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.368362 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeHasSufficientPID" Jul 08 15:50:55.406323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.406293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bb0963c382b2442e093109581c52883-config\") pod \"kube-apiserver-proxy-ip-10-0-136-198.ec2.internal\" (UID: \"4bb0963c382b2442e093109581c52883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.406323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.406317 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.406447 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.406335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.409196 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.409180 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-198.ec2.internal\" not found" node="ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.413618 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.413603 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-198.ec2.internal\" not found" node="ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.415606 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.415592 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.506878 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.506878 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.507037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bb0963c382b2442e093109581c52883-config\") pod \"kube-apiserver-proxy-ip-10-0-136-198.ec2.internal\" (UID: \"4bb0963c382b2442e093109581c52883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.507037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506923 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bb0963c382b2442e093109581c52883-config\") pod \"kube-apiserver-proxy-ip-10-0-136-198.ec2.internal\" (UID: \"4bb0963c382b2442e093109581c52883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.507037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506950 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.507037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.506968 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf58a048dec5ee7c100028553997da99-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal\" (UID: \"bf58a048dec5ee7c100028553997da99\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.515978 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.515950 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.616787 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.616733 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.713233 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.713213 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.717137 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.717054 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.717137 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:55.717088 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:55.818094 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.818065 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:55.918603 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:55.918523 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:56.019119 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:56.019090 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:56.047189 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.047161 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 15:50:56.047655 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.047281 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 15:50:56.120135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:56.120106 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-198.ec2.internal\" not found" Jul 08 15:50:56.155235 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.155196 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 15:45:55 +0000 UTC" deadline="2028-01-21 14:17:20.408509234 +0000 UTC" Jul 08 15:50:56.155235 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.155227 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13486h26m24.253284902s" Jul 08 15:50:56.215183 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.215163 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 15:50:56.221591 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:56.221534 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb0963c382b2442e093109581c52883.slice/crio-97ebd210d00f56783d9874ce1587a2bdd82537c84b543f7b878cde6659dfcc7e WatchSource:0}: Error finding container 97ebd210d00f56783d9874ce1587a2bdd82537c84b543f7b878cde6659dfcc7e: Status 404 returned error can't find the container with id 97ebd210d00f56783d9874ce1587a2bdd82537c84b543f7b878cde6659dfcc7e Jul 08 15:50:56.225592 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:56.224480 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf58a048dec5ee7c100028553997da99.slice/crio-073564193ebe3c81741bf87473da49dff051c6847bbf8a3185759d02a3147d56 WatchSource:0}: Error finding container 073564193ebe3c81741bf87473da49dff051c6847bbf8a3185759d02a3147d56: Status 404 returned error can't find the container with id 073564193ebe3c81741bf87473da49dff051c6847bbf8a3185759d02a3147d56 Jul 08 15:50:56.228325 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.228308 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 15:50:56.265780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.265738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" event={"ID":"bf58a048dec5ee7c100028553997da99","Type":"ContainerStarted","Data":"073564193ebe3c81741bf87473da49dff051c6847bbf8a3185759d02a3147d56"} Jul 08 15:50:56.266693 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.266674 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" event={"ID":"4bb0963c382b2442e093109581c52883","Type":"ContainerStarted","Data":"97ebd210d00f56783d9874ce1587a2bdd82537c84b543f7b878cde6659dfcc7e"} Jul 08 15:50:56.304858 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.304839 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" Jul 08 15:50:56.317553 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.317535 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 15:50:56.320496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.320484 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" Jul 08 15:50:56.329113 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.329098 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 15:50:56.464431 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.464374 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 15:50:56.619395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:56.619368 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 15:50:57.087562 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.087528 2566 apiserver.go:52] "Watching apiserver" Jul 08 15:50:57.097437 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.097413 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 15:50:57.098211 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.097859 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-k5np9","kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw","openshift-cluster-node-tuning-operator/tuned-jgpfg","openshift-dns/node-resolver-h6n7j","openshift-image-registry/node-ca-drs65","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal","openshift-multus/multus-additional-cni-plugins-pf49f","openshift-multus/multus-htrtz","kube-system/global-pull-secret-syncer-tczlz","kube-system/konnectivity-agent-pshm9","openshift-multus/network-metrics-daemon-772cg","openshift-network-diagnostics/network-check-target-vlnjs","openshift-network-operator/iptables-alerter-wv5fq"] Jul 08 15:50:57.099648 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.099626 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.099750 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.099712 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:50:57.101362 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.101345 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:57.101446 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.101399 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:50:57.103011 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.102719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.104647 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.103997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.105110 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.105780 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106127 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ptc59\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106364 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106696 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106822 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106925 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fs7hm\"" Jul 08 15:50:57.107548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.106983 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 15:50:57.109113 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.109094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.112240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.111025 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.112240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.111590 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l7cpz\"" Jul 08 15:50:57.112240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.111609 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.112240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.111840 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.112240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.111853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.113090 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.113073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.113932 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.113912 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.114068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.114051 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 15:50:57.114373 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.114355 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.114887 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.114871 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jbctp\"" Jul 08 15:50:57.115315 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115297 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.115414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-device-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.115414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-systemd\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr796\" (UniqueName: \"kubernetes.io/projected/5805accd-d0a5-47b9-800e-e31df2342fb5-kube-api-access-rr796\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.115414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-run\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-sys\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-kubelet-config\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysconfig\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-conf\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-lib-modules\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-dbus\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-var-lib-kubelet\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.115629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-host\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.116078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dvv\" (UniqueName: \"kubernetes.io/projected/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-kube-api-access-54dvv\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.116078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115784 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.116078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-tmp\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.116078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.115840 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-tuned\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.117267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.117336 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117282 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-modprobe-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.117336 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-kubernetes\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.117442 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.117442 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-tmp-dir\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.117442 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj66m\" (UniqueName: \"kubernetes.io/projected/e5050ed6-e97a-4832-baf0-5e171b85f0ad-kube-api-access-vj66m\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.117586 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.117586 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:57.117586 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.117748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.117748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117620 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l56q4\"" Jul 08 15:50:57.117748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-hosts-file\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.117748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.116648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-47f2m\"" Jul 08 15:50:57.117963 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.116921 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.118317 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.118297 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.118436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117001 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 15:50:57.118604 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.118588 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 15:50:57.118742 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.118722 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 15:50:57.118918 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117255 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.119042 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.117173 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.119127 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.119110 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 15:50:57.119184 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.119145 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-frhlb\"" Jul 08 15:50:57.119184 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.119164 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.119592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.119573 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 15:50:57.121254 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.121230 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 15:50:57.122483 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.122466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 15:50:57.122578 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.122548 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.122934 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.122960 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.123159 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.123200 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8nnmz\"" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.123261 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.123420 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.123264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 15:50:57.124538 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.124520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.124619 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.124597 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:50:57.125598 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.125437 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 15:50:57.125598 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.125491 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 15:50:57.127276 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.127256 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hv8sq\"" Jul 08 15:50:57.151339 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.151315 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x2p84" Jul 08 15:50:57.161285 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.161261 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x2p84" Jul 08 15:50:57.206223 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.206193 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 15:50:57.218066 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218033 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.218217 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.218217 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx8m6\" (UniqueName: \"kubernetes.io/projected/2a96e133-d26d-4fa8-827c-6dae2afe9e64-kube-api-access-wx8m6\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.218217 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj66m\" (UniqueName: \"kubernetes.io/projected/e5050ed6-e97a-4832-baf0-5e171b85f0ad-kube-api-access-vj66m\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-kubelet\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-var-lib-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwf7g\" (UniqueName: \"kubernetes.io/projected/a2109ac5-4292-4774-9731-c6dccb981dfc-kube-api-access-rwf7g\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-socket-dir-parent\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.218395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-etc-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2109ac5-4292-4774-9731-c6dccb981dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-systemd\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41d09967-e2a6-4950-83dc-b5a41917b035-konnectivity-ca\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-system-cni-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cnibin\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-kubelet\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218639 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr796\" (UniqueName: \"kubernetes.io/projected/5805accd-d0a5-47b9-800e-e31df2342fb5-kube-api-access-rr796\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-netd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218708 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-systemd\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-system-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.218764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-conf\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a472ac-8359-42de-8fb2-3fae3c6117c0-host\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-host\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54dvv\" (UniqueName: \"kubernetes.io/projected/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-kube-api-access-54dvv\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218964 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-conf\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-host\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.218981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-node-log\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rtp\" (UniqueName: \"kubernetes.io/projected/0ac03da3-cc5b-498c-b834-11719d2f4a21-kube-api-access-j7rtp\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02022139-aa81-43d2-a1e2-8c5f3859b229-iptables-alerter-script\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-systemd-units\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-netns\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219319 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-ovn\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-k8s-cni-cncf-io\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.219450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-modprobe-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-kubernetes\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-tmp-dir\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219560 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-modprobe-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-slash\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-systemd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4b9p\" (UniqueName: \"kubernetes.io/projected/91a472ac-8359-42de-8fb2-3fae3c6117c0-kube-api-access-c4b9p\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-os-release\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219796 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t8ns\" (UniqueName: \"kubernetes.io/projected/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-kube-api-access-4t8ns\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219822 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cnibin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-os-release\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219889 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-tmp-dir\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cni-binary-copy\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-multus\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219951 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-kubernetes\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.220352 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.219998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-hosts-file\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.220053 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-bin\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220135 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-hostroot\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-hosts-file\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-conf-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.220242 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:50:57.720208263 +0000 UTC m=+2.983226933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-daemon-config\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02022139-aa81-43d2-a1e2-8c5f3859b229-host-slash\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-device-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220481 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-device-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-env-overrides\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-netns\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-run\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-sys\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-etc-kubernetes\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220764 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-kubelet-config\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220779 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysconfig\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-lib-modules\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-script-lib\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220840 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-bin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7xfg\" (UniqueName: \"kubernetes.io/projected/02022139-aa81-43d2-a1e2-8c5f3859b229-kube-api-access-w7xfg\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-dbus\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-var-lib-kubelet\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-config\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.220977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41d09967-e2a6-4950-83dc-b5a41917b035-agent-certs\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221007 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-multus-certs\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-tmp\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a472ac-8359-42de-8fb2-3fae3c6117c0-serviceca\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.221885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-tuned\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-log-socket\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221279 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-run\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221331 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-sys\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221371 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-kubelet-config\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221405 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysconfig\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221481 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-lib-modules\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221583 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48789b43-33de-4682-aeb5-06bbde0259e9-dbus\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221619 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.221669 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-var-lib-kubelet\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.222592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.222024 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 15:50:57.223236 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.223184 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5805accd-d0a5-47b9-800e-e31df2342fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.224182 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.224155 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-sysctl-d\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.224987 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.224964 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:50:57.225092 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.224995 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:50:57.225092 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.225008 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:57.225092 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.225071 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:50:57.72505258 +0000 UTC m=+2.988071243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:57.227343 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.227262 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-tmp\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.227343 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.227301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5050ed6-e97a-4832-baf0-5e171b85f0ad-etc-tuned\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.230173 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.230002 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr796\" (UniqueName: \"kubernetes.io/projected/5805accd-d0a5-47b9-800e-e31df2342fb5-kube-api-access-rr796\") pod \"aws-ebs-csi-driver-node-58rxw\" (UID: \"5805accd-d0a5-47b9-800e-e31df2342fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.230173 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.230132 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj66m\" (UniqueName: \"kubernetes.io/projected/e5050ed6-e97a-4832-baf0-5e171b85f0ad-kube-api-access-vj66m\") pod \"tuned-jgpfg\" (UID: \"e5050ed6-e97a-4832-baf0-5e171b85f0ad\") " pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.230759 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.230702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dvv\" (UniqueName: \"kubernetes.io/projected/8ead0a15-65c5-47a6-b1b6-10bdf1b92031-kube-api-access-54dvv\") pod \"node-resolver-h6n7j\" (UID: \"8ead0a15-65c5-47a6-b1b6-10bdf1b92031\") " pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.321799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-systemd-units\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.321799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-netns\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.321799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-ovn\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-k8s-cni-cncf-io\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-slash\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321826 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-systemd-units\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321871 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-systemd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321900 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-systemd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-netns\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4b9p\" (UniqueName: \"kubernetes.io/projected/91a472ac-8359-42de-8fb2-3fae3c6117c0-kube-api-access-c4b9p\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321958 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-ovn\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.321962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-os-release\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322034 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8ns\" (UniqueName: \"kubernetes.io/projected/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-kube-api-access-4t8ns\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cnibin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322085 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-os-release\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322111 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cni-binary-copy\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-multus\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-bin\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322214 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-hostroot\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-conf-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-daemon-config\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322273 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-k8s-cni-cncf-io\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02022139-aa81-43d2-a1e2-8c5f3859b229-host-slash\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-env-overrides\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-netns\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-os-release\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322427 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-slash\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-etc-kubernetes\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322453 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.322856 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-script-lib\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322492 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-bin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7xfg\" (UniqueName: \"kubernetes.io/projected/02022139-aa81-43d2-a1e2-8c5f3859b229-kube-api-access-w7xfg\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-config\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41d09967-e2a6-4950-83dc-b5a41917b035-agent-certs\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322579 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-bin\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322603 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-multus-certs\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a472ac-8359-42de-8fb2-3fae3c6117c0-serviceca\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-log-socket\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322712 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx8m6\" (UniqueName: \"kubernetes.io/projected/2a96e133-d26d-4fa8-827c-6dae2afe9e64-kube-api-access-wx8m6\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-kubelet\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-var-lib-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwf7g\" (UniqueName: \"kubernetes.io/projected/a2109ac5-4292-4774-9731-c6dccb981dfc-kube-api-access-rwf7g\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322860 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-socket-dir-parent\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322884 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cnibin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-etc-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.323719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322919 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-etc-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2109ac5-4292-4774-9731-c6dccb981dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322947 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-env-overrides\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322986 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-hostroot\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323009 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-etc-kubernetes\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02022139-aa81-43d2-a1e2-8c5f3859b229-host-slash\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323024 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-script-lib\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323033 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-conf-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-os-release\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323148 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-bin\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.323179 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.322398 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.323243 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:50:57.823227581 +0000 UTC m=+3.086246245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41d09967-e2a6-4950-83dc-b5a41917b035-konnectivity-ca\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323279 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-var-lib-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.324683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-system-cni-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cnibin\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-kubelet\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-netd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-system-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323525 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323591 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a472ac-8359-42de-8fb2-3fae3c6117c0-host\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-node-log\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rtp\" (UniqueName: \"kubernetes.io/projected/0ac03da3-cc5b-498c-b834-11719d2f4a21-kube-api-access-j7rtp\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323734 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-system-cni-dir\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323838 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-daemon-config\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.323927 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-system-cni-dir\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-cni-netd\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324143 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-log-socket\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.325496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324268 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-run-openvswitch\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324311 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a96e133-d26d-4fa8-827c-6dae2afe9e64-cni-binary-copy\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324313 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a472ac-8359-42de-8fb2-3fae3c6117c0-serviceca\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324367 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-node-log\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02022139-aa81-43d2-a1e2-8c5f3859b229-iptables-alerter-script\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324683 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324875 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-multus-socket-dir-parent\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324940 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-multus-certs\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.324989 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-cnibin\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325021 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41d09967-e2a6-4950-83dc-b5a41917b035-konnectivity-ca\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-cni-multus\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325089 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2109ac5-4292-4774-9731-c6dccb981dfc-host-kubelet\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325213 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-var-lib-kubelet\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325303 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02022139-aa81-43d2-a1e2-8c5f3859b229-iptables-alerter-script\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325351 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a96e133-d26d-4fa8-827c-6dae2afe9e64-host-run-netns\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325429 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a472ac-8359-42de-8fb2-3fae3c6117c0-host\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.326379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.325975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2109ac5-4292-4774-9731-c6dccb981dfc-ovnkube-config\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.327865 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.327843 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2109ac5-4292-4774-9731-c6dccb981dfc-ovn-node-metrics-cert\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.328031 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.328008 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41d09967-e2a6-4950-83dc-b5a41917b035-agent-certs\") pod \"konnectivity-agent-pshm9\" (UID: \"41d09967-e2a6-4950-83dc-b5a41917b035\") " pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.331630 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.331605 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8ns\" (UniqueName: \"kubernetes.io/projected/c84d0b88-ac06-40b3-b61c-9cfa46f1aad4-kube-api-access-4t8ns\") pod \"multus-additional-cni-plugins-pf49f\" (UID: \"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4\") " pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.332952 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.332927 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4b9p\" (UniqueName: \"kubernetes.io/projected/91a472ac-8359-42de-8fb2-3fae3c6117c0-kube-api-access-c4b9p\") pod \"node-ca-drs65\" (UID: \"91a472ac-8359-42de-8fb2-3fae3c6117c0\") " pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.334704 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.334676 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7xfg\" (UniqueName: \"kubernetes.io/projected/02022139-aa81-43d2-a1e2-8c5f3859b229-kube-api-access-w7xfg\") pod \"iptables-alerter-wv5fq\" (UID: \"02022139-aa81-43d2-a1e2-8c5f3859b229\") " pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.335145 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.335120 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx8m6\" (UniqueName: \"kubernetes.io/projected/2a96e133-d26d-4fa8-827c-6dae2afe9e64-kube-api-access-wx8m6\") pod \"multus-htrtz\" (UID: \"2a96e133-d26d-4fa8-827c-6dae2afe9e64\") " pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.336748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.336724 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwf7g\" (UniqueName: \"kubernetes.io/projected/a2109ac5-4292-4774-9731-c6dccb981dfc-kube-api-access-rwf7g\") pod \"ovnkube-node-k5np9\" (UID: \"a2109ac5-4292-4774-9731-c6dccb981dfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.337005 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.336972 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rtp\" (UniqueName: \"kubernetes.io/projected/0ac03da3-cc5b-498c-b834-11719d2f4a21-kube-api-access-j7rtp\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.423628 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.423571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" Jul 08 15:50:57.430801 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.430765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" Jul 08 15:50:57.434615 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.434365 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5805accd_d0a5_47b9_800e_e31df2342fb5.slice/crio-b0662c59740d573a73aa8a2ec85f6fb69fa17feb0e49d7c7d41e52d6439c60ac WatchSource:0}: Error finding container b0662c59740d573a73aa8a2ec85f6fb69fa17feb0e49d7c7d41e52d6439c60ac: Status 404 returned error can't find the container with id b0662c59740d573a73aa8a2ec85f6fb69fa17feb0e49d7c7d41e52d6439c60ac Jul 08 15:50:57.440755 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.440727 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5050ed6_e97a_4832_baf0_5e171b85f0ad.slice/crio-b8ccca8fa81a907a73bcc410ed30091f455af191ac333c29f162856e42009f4a WatchSource:0}: Error finding container b8ccca8fa81a907a73bcc410ed30091f455af191ac333c29f162856e42009f4a: Status 404 returned error can't find the container with id b8ccca8fa81a907a73bcc410ed30091f455af191ac333c29f162856e42009f4a Jul 08 15:50:57.442223 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.441879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h6n7j" Jul 08 15:50:57.449839 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.449816 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-drs65" Jul 08 15:50:57.451523 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.451192 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ead0a15_65c5_47a6_b1b6_10bdf1b92031.slice/crio-7bfa463662308c6ef4fe9f47e3026bfc4a1efb3379fbca24e44562973fdbeb36 WatchSource:0}: Error finding container 7bfa463662308c6ef4fe9f47e3026bfc4a1efb3379fbca24e44562973fdbeb36: Status 404 returned error can't find the container with id 7bfa463662308c6ef4fe9f47e3026bfc4a1efb3379fbca24e44562973fdbeb36 Jul 08 15:50:57.456853 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.456832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htrtz" Jul 08 15:50:57.460326 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.460301 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a472ac_8359_42de_8fb2_3fae3c6117c0.slice/crio-e6120cf8e85058be4e70e0f8ba5f533d845df398288b5a8213a0e943b0e9ac83 WatchSource:0}: Error finding container e6120cf8e85058be4e70e0f8ba5f533d845df398288b5a8213a0e943b0e9ac83: Status 404 returned error can't find the container with id e6120cf8e85058be4e70e0f8ba5f533d845df398288b5a8213a0e943b0e9ac83 Jul 08 15:50:57.466470 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.466434 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a96e133_d26d_4fa8_827c_6dae2afe9e64.slice/crio-f44fd1c1e03f4c9f8adb2145ceb1c6e3dc2152de9e1a1f4fe8d0cef3c5041dd3 WatchSource:0}: Error finding container f44fd1c1e03f4c9f8adb2145ceb1c6e3dc2152de9e1a1f4fe8d0cef3c5041dd3: Status 404 returned error can't find the container with id f44fd1c1e03f4c9f8adb2145ceb1c6e3dc2152de9e1a1f4fe8d0cef3c5041dd3 Jul 08 15:50:57.467378 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.467362 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wv5fq" Jul 08 15:50:57.468519 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.468501 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 15:50:57.475355 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:57.475328 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02022139_aa81_43d2_a1e2_8c5f3859b229.slice/crio-7d986d51f581cff8f8e814f3a2be8c8d9ad2fe7bcfb25be560ed994d376d2bac WatchSource:0}: Error finding container 7d986d51f581cff8f8e814f3a2be8c8d9ad2fe7bcfb25be560ed994d376d2bac: Status 404 returned error can't find the container with id 7d986d51f581cff8f8e814f3a2be8c8d9ad2fe7bcfb25be560ed994d376d2bac Jul 08 15:50:57.478842 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.478825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pf49f" Jul 08 15:50:57.485499 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.485478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:50:57.493898 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.493878 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:50:57.728845 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.728755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:57.729015 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.728861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:57.729015 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.728944 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:57.729135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.729017 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:50:58.728994506 +0000 UTC m=+3.992013175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:57.729135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.729031 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:50:57.729135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.729053 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:50:57.729135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.729067 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:57.729135 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.729120 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:50:58.729103604 +0000 UTC m=+3.992122256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:57.829834 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:57.829799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:57.830022 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.830003 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:57.830081 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:57.830074 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:50:58.830053473 +0000 UTC m=+4.093072125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:58.163802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.163722 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 15:45:57 +0000 UTC" deadline="2027-12-25 14:05:34.451965914 +0000 UTC" Jul 08 15:50:58.163802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.163753 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12838h14m36.288216849s" Jul 08 15:50:58.263408 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.263367 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:58.263572 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.263503 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:50:58.270907 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.270868 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h6n7j" event={"ID":"8ead0a15-65c5-47a6-b1b6-10bdf1b92031","Type":"ContainerStarted","Data":"7bfa463662308c6ef4fe9f47e3026bfc4a1efb3379fbca24e44562973fdbeb36"} Jul 08 15:50:58.272184 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.272147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" event={"ID":"e5050ed6-e97a-4832-baf0-5e171b85f0ad","Type":"ContainerStarted","Data":"b8ccca8fa81a907a73bcc410ed30091f455af191ac333c29f162856e42009f4a"} Jul 08 15:50:58.273378 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.273350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" event={"ID":"5805accd-d0a5-47b9-800e-e31df2342fb5","Type":"ContainerStarted","Data":"b0662c59740d573a73aa8a2ec85f6fb69fa17feb0e49d7c7d41e52d6439c60ac"} Jul 08 15:50:58.274624 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.274601 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wv5fq" event={"ID":"02022139-aa81-43d2-a1e2-8c5f3859b229","Type":"ContainerStarted","Data":"7d986d51f581cff8f8e814f3a2be8c8d9ad2fe7bcfb25be560ed994d376d2bac"} Jul 08 15:50:58.275739 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.275716 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htrtz" event={"ID":"2a96e133-d26d-4fa8-827c-6dae2afe9e64","Type":"ContainerStarted","Data":"f44fd1c1e03f4c9f8adb2145ceb1c6e3dc2152de9e1a1f4fe8d0cef3c5041dd3"} Jul 08 15:50:58.276886 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.276842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-drs65" event={"ID":"91a472ac-8359-42de-8fb2-3fae3c6117c0","Type":"ContainerStarted","Data":"e6120cf8e85058be4e70e0f8ba5f533d845df398288b5a8213a0e943b0e9ac83"} Jul 08 15:50:58.737561 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.737470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:50:58.737780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.737563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:58.737780 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.737737 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:50:58.737780 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.737763 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:50:58.737941 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.737808 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:58.738017 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.738000 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:51:00.737913682 +0000 UTC m=+6.000932348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:50:58.738424 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.738400 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:58.738523 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.738459 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:00.738444755 +0000 UTC m=+6.001463420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:50:58.838411 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:58.838317 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:58.838604 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.838551 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:58.838676 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:58.838635 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:00.838614977 +0000 UTC m=+6.101633645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:50:58.906474 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:50:58.905566 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84d0b88_ac06_40b3_b61c_9cfa46f1aad4.slice/crio-8a2553eddaf10b2df2a819381b2c3499b5ce1d3ba21876abba61b713dc2435dc WatchSource:0}: Error finding container 8a2553eddaf10b2df2a819381b2c3499b5ce1d3ba21876abba61b713dc2435dc: Status 404 returned error can't find the container with id 8a2553eddaf10b2df2a819381b2c3499b5ce1d3ba21876abba61b713dc2435dc Jul 08 15:50:59.165027 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.164746 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 15:45:57 +0000 UTC" deadline="2028-04-21 05:35:32.03014095 +0000 UTC" Jul 08 15:50:59.165388 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.165027 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15661h44m32.865118712s" Jul 08 15:50:59.264553 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.263784 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:50:59.264553 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:59.263936 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:50:59.264553 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.264364 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:50:59.264553 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:50:59.264456 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:50:59.292545 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.292504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerStarted","Data":"8a2553eddaf10b2df2a819381b2c3499b5ce1d3ba21876abba61b713dc2435dc"} Jul 08 15:50:59.297490 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.297458 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pshm9" event={"ID":"41d09967-e2a6-4950-83dc-b5a41917b035","Type":"ContainerStarted","Data":"d9cd08ca8da0cc134d98fb2298c6123c4a871a6e3dd5a530ed78526ad4533953"} Jul 08 15:50:59.307552 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.307521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" event={"ID":"4bb0963c382b2442e093109581c52883","Type":"ContainerStarted","Data":"f3fd32500f2b66a51cfc0e56753c5b4ad27fbf7c8c848d7645bc732052f55c68"} Jul 08 15:50:59.309683 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:50:59.309641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"b397b635d017568a04c6b31307a684a59bb9352790eec56ec4ae8632cbe4626b"} Jul 08 15:51:00.263754 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.263719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:00.264261 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.263866 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:00.323516 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.323479 2566 generic.go:356] "Generic (PLEG): container finished" podID="bf58a048dec5ee7c100028553997da99" containerID="18352062097a144d80c4a6b4253a76024bf3fe15b53b84858fd84befe6c88ed3" exitCode=0 Jul 08 15:51:00.324490 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.324460 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" event={"ID":"bf58a048dec5ee7c100028553997da99","Type":"ContainerDied","Data":"18352062097a144d80c4a6b4253a76024bf3fe15b53b84858fd84befe6c88ed3"} Jul 08 15:51:00.343116 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.343045 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-198.ec2.internal" podStartSLOduration=3.343024444 podStartE2EDuration="3.343024444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:50:59.324826904 +0000 UTC m=+4.587845582" watchObservedRunningTime="2026-07-08 15:51:00.343024444 +0000 UTC m=+5.606043119" Jul 08 15:51:00.756213 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.756089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:00.756213 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.756187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:00.756423 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756345 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:51:00.756423 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756365 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:51:00.756423 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756377 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:00.756580 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756441 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:51:04.756419868 +0000 UTC m=+10.019438533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:00.756934 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756911 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:00.757034 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.756970 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:04.756954723 +0000 UTC m=+10.019973372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:00.857606 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:00.857033 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:00.857606 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.857170 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:00.857606 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:00.857257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:04.857234985 +0000 UTC m=+10.120253639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:01.264738 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:01.264705 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:01.265205 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:01.264841 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:01.265205 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:01.264709 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:01.265338 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:01.265255 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:02.263587 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:02.263535 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:02.263762 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:02.263709 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:03.263743 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:03.263704 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:03.264235 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:03.263861 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:03.264235 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:03.263990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:03.264235 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:03.264119 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:04.264155 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:04.263709 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:04.264155 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.263855 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:04.794104 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:04.794062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:04.794298 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:04.794136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:04.794298 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794272 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:04.794391 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794333 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:12.794311809 +0000 UTC m=+18.057330458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:04.794817 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794798 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:51:04.794924 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794821 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:51:04.794924 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794831 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:04.794924 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.794871 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:51:12.794855892 +0000 UTC m=+18.057874541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:04.894959 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:04.894917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:04.895151 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.895102 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:04.895228 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:04.895173 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:12.895153056 +0000 UTC m=+18.158171710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:05.263955 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:05.263866 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:05.264095 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:05.264008 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:05.264370 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:05.264354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:05.264759 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:05.264439 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:06.264144 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:06.263676 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:06.264144 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:06.263821 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:07.264245 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:07.264212 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:07.264757 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:07.264212 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:07.264757 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:07.264351 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:07.264757 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:07.264471 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:08.263425 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:08.263393 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:08.263651 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:08.263488 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:09.263064 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:09.263040 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:09.263431 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:09.263081 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:09.263431 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:09.263159 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:09.263431 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:09.263246 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:10.263078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.262880 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:10.263541 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:10.263172 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:10.351144 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.351062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" event={"ID":"bf58a048dec5ee7c100028553997da99","Type":"ContainerStarted","Data":"293e495a5d0cf578ceb78a87bb30f0445301becf521561065005185812e4fcde"} Jul 08 15:51:10.352918 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.352892 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-drs65" event={"ID":"91a472ac-8359-42de-8fb2-3fae3c6117c0","Type":"ContainerStarted","Data":"ed21aded5a734161d89f31bdd9fd905847f46193aa9f81ebe23fb0162209bc0e"} Jul 08 15:51:10.354912 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.354889 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" event={"ID":"5805accd-d0a5-47b9-800e-e31df2342fb5","Type":"ContainerStarted","Data":"f38096cbc9b0ddfd83759a82218eba689da2c7fb647c3dd732dc480bab97ae1e"} Jul 08 15:51:10.356434 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.356408 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="3bc5f8b651ad90f0b26a5ece7b138f62d7be88ad1f4b964cca7019d7f2a44244" exitCode=0 Jul 08 15:51:10.356554 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.356495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"3bc5f8b651ad90f0b26a5ece7b138f62d7be88ad1f4b964cca7019d7f2a44244"} Jul 08 15:51:10.358688 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.358663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pshm9" event={"ID":"41d09967-e2a6-4950-83dc-b5a41917b035","Type":"ContainerStarted","Data":"c96038375572094671b540d5de2735008eec47de1eff394b1f4f2ffc67301610"} Jul 08 15:51:10.360159 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.360127 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h6n7j" event={"ID":"8ead0a15-65c5-47a6-b1b6-10bdf1b92031","Type":"ContainerStarted","Data":"50eff9057bf82ce31ca0cd00fe85e3e82af44b1d5bf3bf3f591cdfdfdaf1f03b"} Jul 08 15:51:10.361696 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.361673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" event={"ID":"e5050ed6-e97a-4832-baf0-5e171b85f0ad","Type":"ContainerStarted","Data":"38ce7169ed75cf6190c69b8eb40a08c2ca24afc61e7fc903b9632ee668065a74"} Jul 08 15:51:10.369121 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.369078 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-198.ec2.internal" podStartSLOduration=13.369061328 podStartE2EDuration="13.369061328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:10.368680757 +0000 UTC m=+15.631699509" watchObservedRunningTime="2026-07-08 15:51:10.369061328 +0000 UTC m=+15.632079999" Jul 08 15:51:10.386730 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.386693 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pshm9" podStartSLOduration=4.031133703 podStartE2EDuration="14.386678758s" totalImagesPullingTime="10.355545055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:10.385961452 +0000 UTC m=+15.648980121" watchObservedRunningTime="2026-07-08 15:51:10.386678758 +0000 UTC m=+15.649697426" Jul 08 15:51:10.407542 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.407487 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jgpfg" podStartSLOduration=2.566000506 podStartE2EDuration="14.407423577s" totalImagesPullingTime="11.841423071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:10.406124881 +0000 UTC m=+15.669143532" watchObservedRunningTime="2026-07-08 15:51:10.407423577 +0000 UTC m=+15.670442246" Jul 08 15:51:10.422240 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.422194 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h6n7j" podStartSLOduration=2.587446633 podStartE2EDuration="14.42217807s" totalImagesPullingTime="11.834731437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:10.421322645 +0000 UTC m=+15.684341352" watchObservedRunningTime="2026-07-08 15:51:10.42217807 +0000 UTC m=+15.685196740" Jul 08 15:51:10.436986 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:10.436945 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-drs65" podStartSLOduration=2.617473355 podStartE2EDuration="14.43693121s" totalImagesPullingTime="11.819457855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:10.436908949 +0000 UTC m=+15.699927620" watchObservedRunningTime="2026-07-08 15:51:10.43693121 +0000 UTC m=+15.699949873" Jul 08 15:51:11.120785 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.120711 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:51:11.121507 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.121321 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:51:11.263555 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.263515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:11.263555 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.263541 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:11.264142 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:11.263657 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:11.264142 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:11.263783 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:11.365399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.365361 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wv5fq" event={"ID":"02022139-aa81-43d2-a1e2-8c5f3859b229","Type":"ContainerStarted","Data":"3c42deecd142fcd65859a92488cc4c3983b7737e774886b178042023af44cbff"} Jul 08 15:51:11.383643 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.383544 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wv5fq" podStartSLOduration=3.601781926 podStartE2EDuration="15.383526728s" totalImagesPullingTime="11.781744802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:11.383519931 +0000 UTC m=+16.646538604" watchObservedRunningTime="2026-07-08 15:51:11.383526728 +0000 UTC m=+16.646545398" Jul 08 15:51:11.774998 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.774922 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:51:11.775849 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:11.775814 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pshm9" Jul 08 15:51:12.263376 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:12.263304 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:12.263517 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.263431 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:12.864024 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:12.863993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:12.864043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864150 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864164 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864184 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864198 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864215 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:28.864196499 +0000 UTC m=+34.127215147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:12.864503 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.864232 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:51:28.864222735 +0000 UTC m=+34.127241382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:12.965194 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:12.965151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:12.965336 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.965252 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:12.965336 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:12.965316 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:28.965299453 +0000 UTC m=+34.228318102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:13.263667 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:13.263595 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:13.263841 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:13.263603 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:13.263912 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:13.263844 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:13.263912 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:13.263867 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:14.263349 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:14.263312 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:14.263909 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:14.263444 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:15.264142 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:15.264112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:15.264596 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:15.264222 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:15.264596 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:15.264280 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:15.264596 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:15.264400 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:16.262937 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:16.262772 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:16.262937 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:16.262904 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:17.263105 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:17.263059 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:17.263526 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:17.263191 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:17.263791 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:17.263651 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:17.263791 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:17.263754 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:18.263806 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:18.263763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:18.264459 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:18.263909 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:19.263346 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:19.263099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:19.263505 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:19.263165 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:19.263505 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:19.263422 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:19.263505 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:19.263493 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:20.263019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:20.262981 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:20.263447 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:20.263100 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:20.714517 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:20.714490 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 15:51:21.172525 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.172445 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T15:51:20.714512911Z","UUID":"92c22422-ccf0-4c10-bac1-d130068bbad5","Handler":null,"Name":"","Endpoint":""} Jul 08 15:51:21.174077 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.174056 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 15:51:21.174173 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.174088 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 15:51:21.263816 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.263795 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:21.264212 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:21.263914 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:21.264212 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.263998 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:21.264212 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:21.264123 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:21.388984 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.388916 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:51:21.389345 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389324 2566 generic.go:356] "Generic (PLEG): container finished" podID="a2109ac5-4292-4774-9731-c6dccb981dfc" containerID="4ed26150c2e29adcb5a217b29679beb0402441bdfbac9b452fd54cf115abc1d8" exitCode=1 Jul 08 15:51:21.389422 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"9f1f08e634c30f3d2f80eab7c17d4703d3fa88d38e6f19ce17d31f602bb472ae"} Jul 08 15:51:21.389482 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389464 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"4319c89f6b2e9d3a0fedc6bf3007b2ee036ab0ad06be7c04d4cd6bbba7934288"} Jul 08 15:51:21.389534 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389500 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"9dd98f60e914a0ad68a47e143f1798b8be044977e2686d68a7a008796ccea5f8"} Jul 08 15:51:21.389534 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389528 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"c95d7c0f6db1018214ab03809ff6153e415e8285bffb05628927e709e3a6bf88"} Jul 08 15:51:21.389665 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerDied","Data":"4ed26150c2e29adcb5a217b29679beb0402441bdfbac9b452fd54cf115abc1d8"} Jul 08 15:51:21.389665 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.389654 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"0cae8dd27adab53da282c165a5f21e8a81d516f81913637e1b96f3c455c6c657"} Jul 08 15:51:21.391528 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.391493 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" event={"ID":"5805accd-d0a5-47b9-800e-e31df2342fb5","Type":"ContainerStarted","Data":"6540a82be9b395e5fec2b31355e0c7ef45901f67d130ff32924133dce1ded6d1"} Jul 08 15:51:21.393414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.393382 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="57f6013673690b36700d4c7aac68bc7209becfe7da81b0404802ca92ba4cf697" exitCode=0 Jul 08 15:51:21.393646 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.393620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"57f6013673690b36700d4c7aac68bc7209becfe7da81b0404802ca92ba4cf697"} Jul 08 15:51:21.395367 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.395345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htrtz" event={"ID":"2a96e133-d26d-4fa8-827c-6dae2afe9e64","Type":"ContainerStarted","Data":"98aa38572dabe464a74fe3bb4528d40c4e5d022580848ab444e66f8352c0ca6d"} Jul 08 15:51:21.435780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:21.435739 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-htrtz" podStartSLOduration=2.342214702 podStartE2EDuration="25.435723674s" totalImagesPullingTime="23.093508972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:21.435668925 +0000 UTC m=+26.698687594" watchObservedRunningTime="2026-07-08 15:51:21.435723674 +0000 UTC m=+26.698742345" Jul 08 15:51:22.263220 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:22.263199 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:22.263306 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:22.263287 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:22.399771 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:22.399727 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" event={"ID":"5805accd-d0a5-47b9-800e-e31df2342fb5","Type":"ContainerStarted","Data":"6482e653e1dd51c1872c18a849137540f16aa55292d595d2141d86611e0fda69"} Jul 08 15:51:22.418899 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:22.418855 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58rxw" podStartSLOduration=2.129045371 podStartE2EDuration="26.418840939s" totalImagesPullingTime="24.289795568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:22.41877203 +0000 UTC m=+27.681790700" watchObservedRunningTime="2026-07-08 15:51:22.418840939 +0000 UTC m=+27.681859607" Jul 08 15:51:23.263102 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.263075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:23.263102 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.263096 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:23.263302 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:23.263209 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:23.263358 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:23.263326 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:23.405026 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.405006 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:51:23.405419 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.405390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"e57eed95c15ac9aadf7935c80019e1ec0af91fabb54e7dfa9b491ece0e76d5b7"} Jul 08 15:51:23.407351 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.407326 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="118db1383567bfe402b32419a9641de577c38e37a715214df32c2cc0ace18e2c" exitCode=0 Jul 08 15:51:23.407442 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:23.407409 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"118db1383567bfe402b32419a9641de577c38e37a715214df32c2cc0ace18e2c"} Jul 08 15:51:24.263579 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:24.263554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:24.263773 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:24.263659 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:25.263925 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:25.263850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:25.264368 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:25.263934 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:25.264368 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:25.264017 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:25.264368 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:25.264099 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:25.413189 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:25.413162 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="da7e70a2db1c7131dc69f834fb388a45e8c6fbda35957456bd8cb8d1c4af4b18" exitCode=0 Jul 08 15:51:25.413323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:25.413207 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"da7e70a2db1c7131dc69f834fb388a45e8c6fbda35957456bd8cb8d1c4af4b18"} Jul 08 15:51:26.263696 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.263498 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:26.263899 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:26.263788 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:26.419940 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.419907 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:51:26.420524 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.420306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"cbec1516ee0e34121efa40bb6e95446df48bd8e7175ad3a46cdf2a5cef655885"} Jul 08 15:51:26.420711 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.420691 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:26.420833 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.420815 2566 scope.go:117] "RemoveContainer" containerID="4ed26150c2e29adcb5a217b29679beb0402441bdfbac9b452fd54cf115abc1d8" Jul 08 15:51:26.439286 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:26.439265 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:27.263040 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.263009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:27.263040 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.263028 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:27.263306 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:27.263144 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:27.263306 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:27.263279 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:27.427093 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.427060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:51:27.427560 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.427491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" event={"ID":"a2109ac5-4292-4774-9731-c6dccb981dfc","Type":"ContainerStarted","Data":"907d7381ae92547409018e6a2c58f28ccf36af2e832696cfe93c4966b15dd188"} Jul 08 15:51:27.427800 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.427780 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:27.427955 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.427932 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:27.446550 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.446526 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:27.460174 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.460119 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" podStartSLOduration=9.794016326 podStartE2EDuration="31.460104908s" totalImagesPullingTime="21.666088582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:27.459709144 +0000 UTC m=+32.722727814" watchObservedRunningTime="2026-07-08 15:51:27.460104908 +0000 UTC m=+32.723123576" Jul 08 15:51:27.499534 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.499511 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-772cg"] Jul 08 15:51:27.499708 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.499646 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:27.499770 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:27.499747 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:27.502430 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.502404 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tczlz"] Jul 08 15:51:27.502527 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.502491 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:27.502605 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:27.502563 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:27.504974 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.504955 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlnjs"] Jul 08 15:51:27.505064 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:27.505034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:27.505117 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:27.505100 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:28.884105 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:28.884068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:28.884123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884227 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884227 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884240 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884252 2566 projected.go:196] Error preparing data for projected volume kube-api-access-98sr8 for pod openshift-network-diagnostics/network-check-target-vlnjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884289 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret podName:48789b43-33de-4682-aeb5-06bbde0259e9 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:00.884274561 +0000 UTC m=+66.147293208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret") pod "global-pull-secret-syncer-tczlz" (UID: "48789b43-33de-4682-aeb5-06bbde0259e9") : object "kube-system"/"original-pull-secret" not registered Jul 08 15:51:28.884475 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.884304 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8 podName:2c3f5924-af89-4e1a-a4c3-3b75d64b22fb nodeName:}" failed. No retries permitted until 2026-07-08 15:52:00.884297887 +0000 UTC m=+66.147316534 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-98sr8" (UniqueName: "kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8") pod "network-check-target-vlnjs" (UID: "2c3f5924-af89-4e1a-a4c3-3b75d64b22fb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 15:51:28.985227 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:28.985199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:28.985362 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.985339 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:28.985399 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:28.985393 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs podName:0ac03da3-cc5b-498c-b834-11719d2f4a21 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:00.985378645 +0000 UTC m=+66.248397292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs") pod "network-metrics-daemon-772cg" (UID: "0ac03da3-cc5b-498c-b834-11719d2f4a21") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 15:51:29.263845 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:29.263776 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:29.263845 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:29.263789 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:29.264134 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:29.263871 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:29.264134 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:29.263954 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:29.264134 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:29.263968 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:29.264134 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:29.264025 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:31.263084 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:31.262918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:31.263501 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:31.262927 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:31.263501 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:31.263148 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:31.263501 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:31.262955 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:31.263501 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:31.263222 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:31.263501 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:31.263319 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:32.441335 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:32.441260 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="23c6f52ebfc556338c2cc5e7ec774025a538bc68517c3ef79e45130bab5bb056" exitCode=0 Jul 08 15:51:32.441335 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:32.441303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"23c6f52ebfc556338c2cc5e7ec774025a538bc68517c3ef79e45130bab5bb056"} Jul 08 15:51:33.263665 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:33.263636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:33.263665 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:33.263659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:33.263907 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:33.263735 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:33.263907 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:33.263733 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlnjs" podUID="2c3f5924-af89-4e1a-a4c3-3b75d64b22fb" Jul 08 15:51:33.263907 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:33.263827 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tczlz" podUID="48789b43-33de-4682-aeb5-06bbde0259e9" Jul 08 15:51:33.264072 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:33.263923 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-772cg" podUID="0ac03da3-cc5b-498c-b834-11719d2f4a21" Jul 08 15:51:33.445904 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:33.445874 2566 generic.go:356] "Generic (PLEG): container finished" podID="c84d0b88-ac06-40b3-b61c-9cfa46f1aad4" containerID="a80fec546fd253aecff023e96557b381fbec138a6952b2b67ee1dc1f6cad88ca" exitCode=0 Jul 08 15:51:33.446316 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:33.445916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerDied","Data":"a80fec546fd253aecff023e96557b381fbec138a6952b2b67ee1dc1f6cad88ca"} Jul 08 15:51:34.450758 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.450730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pf49f" event={"ID":"c84d0b88-ac06-40b3-b61c-9cfa46f1aad4","Type":"ContainerStarted","Data":"8bc1a5f3d24ee70c24cef84bf4ee25a66cea9ebd4b820f78e4bef5cb583cb01d"} Jul 08 15:51:34.479367 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.479267 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pf49f" podStartSLOduration=8.683071089 podStartE2EDuration="38.479251979s" totalImagesPullingTime="29.79618089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:34.476372026 +0000 UTC m=+39.739390695" watchObservedRunningTime="2026-07-08 15:51:34.479251979 +0000 UTC m=+39.742270647" Jul 08 15:51:34.525865 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.525843 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-198.ec2.internal" event="NodeReady" Jul 08 15:51:34.525974 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.525945 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 15:51:34.575327 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.575304 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8"] Jul 08 15:51:34.597473 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.597453 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw"] Jul 08 15:51:34.597605 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.597590 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.600445 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.600426 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 15:51:34.600445 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.600438 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.600596 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.600462 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 15:51:34.600866 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.600847 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.601118 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.601098 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2tkzj\"" Jul 08 15:51:34.615888 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.615864 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:51:34.616001 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.615949 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" Jul 08 15:51:34.619697 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.619677 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.619778 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.619677 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.619778 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.619727 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k2l5t\"" Jul 08 15:51:34.625753 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.625724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llbbj\" (UniqueName: \"kubernetes.io/projected/09287bf3-d83b-445e-a731-2e541313c740-kube-api-access-llbbj\") pod \"volume-data-source-validator-5d456fdbcd-cmwrw\" (UID: \"09287bf3-d83b-445e-a731-2e541313c740\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" Jul 08 15:51:34.625753 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.625761 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ad66488-6def-4a0d-8f34-3a8a499f0800-config\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.625915 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.625780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqzm\" (UniqueName: \"kubernetes.io/projected/2ad66488-6def-4a0d-8f34-3a8a499f0800-kube-api-access-gpqzm\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.625915 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.625801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ad66488-6def-4a0d-8f34-3a8a499f0800-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.629425 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.629408 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8"] Jul 08 15:51:34.629491 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.629442 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58"] Jul 08 15:51:34.629550 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.629503 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.637862 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.637657 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 15:51:34.648051 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.648031 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:51:34.648160 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.648060 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw"] Jul 08 15:51:34.648160 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.648079 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58"] Jul 08 15:51:34.648160 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.648101 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c79v7"] Jul 08 15:51:34.648160 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.648146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.649198 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.649162 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rpkvs\"" Jul 08 15:51:34.649198 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.649172 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 15:51:34.649346 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.649218 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 15:51:34.653938 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.653917 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.654018 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.653979 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 15:51:34.654088 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.654051 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 15:51:34.654088 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.654059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nl7kw\"" Jul 08 15:51:34.654302 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.654286 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.654633 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.654614 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 15:51:34.669668 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.669637 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm"] Jul 08 15:51:34.669822 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.669728 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.672878 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.672857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 15:51:34.673013 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.672997 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mjhcc\"" Jul 08 15:51:34.674042 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.674024 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 15:51:34.681635 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.681619 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl"] Jul 08 15:51:34.681743 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.681729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.684765 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.684750 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 15:51:34.685410 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.685397 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.686127 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.686111 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.686182 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.686172 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9cfl2\"" Jul 08 15:51:34.686271 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.686252 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 15:51:34.695749 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.695732 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7j7sj"] Jul 08 15:51:34.695842 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.695828 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.699314 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.699295 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.699402 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.699339 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 15:51:34.699402 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.699345 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6j6qt\"" Jul 08 15:51:34.699770 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.699752 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.707923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.707907 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-klzjd"] Jul 08 15:51:34.708022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.708009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.710304 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.710284 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 15:51:34.710799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.710785 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.710932 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.710918 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 15:51:34.711020 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.711005 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.711243 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.711229 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vgbtm\"" Jul 08 15:51:34.718077 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.718054 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 15:51:34.720071 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.720053 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppj7"] Jul 08 15:51:34.720166 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.720154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.722451 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.722433 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 15:51:34.722725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.722712 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.722762 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.722721 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9cfxv\"" Jul 08 15:51:34.722939 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.722926 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.722998 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.722935 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 15:51:34.726544 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82e36f17-b33c-406c-9648-b0078e7c4184-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.726647 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726559 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.726647 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.726715 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726684 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a33585d-426f-43fc-98c8-2fd6a1e00fef-serving-cert\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.726759 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgc5c\" (UniqueName: \"kubernetes.io/projected/0a33585d-426f-43fc-98c8-2fd6a1e00fef-kube-api-access-dgc5c\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.726800 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726784 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-tmp\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.726863 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.726923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726892 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llbbj\" (UniqueName: \"kubernetes.io/projected/09287bf3-d83b-445e-a731-2e541313c740-kube-api-access-llbbj\") pod \"volume-data-source-validator-5d456fdbcd-cmwrw\" (UID: \"09287bf3-d83b-445e-a731-2e541313c740\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" Jul 08 15:51:34.726975 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ad66488-6def-4a0d-8f34-3a8a499f0800-config\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.726975 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll62z\" (UniqueName: \"kubernetes.io/projected/82e36f17-b33c-406c-9648-b0078e7c4184-kube-api-access-ll62z\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.727076 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.726977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqzm\" (UniqueName: \"kubernetes.io/projected/2ad66488-6def-4a0d-8f34-3a8a499f0800-kube-api-access-gpqzm\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.727076 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-config\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.727076 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-snapshots\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.727076 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ad66488-6def-4a0d-8f34-3a8a499f0800-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.727254 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4b5r\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727254 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727235 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727342 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727290 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727386 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.727386 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727467 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727467 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thqj\" (UniqueName: \"kubernetes.io/projected/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-kube-api-access-9thqj\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.727467 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-trusted-ca\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.727617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scx7z\" (UniqueName: \"kubernetes.io/projected/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-kube-api-access-scx7z\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.727617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727481 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ad66488-6def-4a0d-8f34-3a8a499f0800-config\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.727617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-tmp-dir\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.727617 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727638 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-serving-cert\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d0f68f-8bae-4925-afb4-bc71338d6599-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7xxb\" (UniqueName: \"kubernetes.io/projected/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-kube-api-access-z7xxb\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727685 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.727799 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.727995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6d0f68f-8bae-4925-afb4-bc71338d6599-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.727995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w64lg\" (UniqueName: \"kubernetes.io/projected/d6d0f68f-8bae-4925-afb4-bc71338d6599-kube-api-access-w64lg\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.727995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.727889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-config-volume\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.728370 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.728351 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 15:51:34.731153 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.731134 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ad66488-6def-4a0d-8f34-3a8a499f0800-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.732155 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.732141 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt"] Jul 08 15:51:34.732234 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.732212 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.734511 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.734493 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 15:51:34.734929 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.734908 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-68mn7\"" Jul 08 15:51:34.735068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.734918 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.735188 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.735147 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.737079 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.737050 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqzm\" (UniqueName: \"kubernetes.io/projected/2ad66488-6def-4a0d-8f34-3a8a499f0800-kube-api-access-gpqzm\") pod \"service-ca-operator-5d46d9ffd4-4q2t8\" (UID: \"2ad66488-6def-4a0d-8f34-3a8a499f0800\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.737333 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.737316 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llbbj\" (UniqueName: \"kubernetes.io/projected/09287bf3-d83b-445e-a731-2e541313c740-kube-api-access-llbbj\") pod \"volume-data-source-validator-5d456fdbcd-cmwrw\" (UID: \"09287bf3-d83b-445e-a731-2e541313c740\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" Jul 08 15:51:34.744317 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.744301 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-bdcbbdb9-r5lfj"] Jul 08 15:51:34.744429 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.744410 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" Jul 08 15:51:34.746645 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.746607 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4wrzz\"" Jul 08 15:51:34.746764 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.746749 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.746918 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.746906 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.756674 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.756657 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn"] Jul 08 15:51:34.756785 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.756770 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.759389 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759368 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 15:51:34.759478 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759413 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 15:51:34.759478 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759424 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 15:51:34.759478 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759469 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 15:51:34.759666 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759483 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 15:51:34.759778 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759760 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pgxtn\"" Jul 08 15:51:34.759883 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.759869 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 15:51:34.769778 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c79v7"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769792 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769825 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7j7sj"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769847 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt"] Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.769860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769859 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-klzjd"] Jul 08 15:51:34.770079 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769870 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bdcbbdb9-r5lfj"] Jul 08 15:51:34.770079 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.769884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppj7"] Jul 08 15:51:34.772253 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.772238 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bcnh2\"" Jul 08 15:51:34.772424 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.772413 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 15:51:34.772721 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.772706 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 15:51:34.828613 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll62z\" (UniqueName: \"kubernetes.io/projected/82e36f17-b33c-406c-9648-b0078e7c4184-kube-api-access-ll62z\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828639 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-config\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-snapshots\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4b5r\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.828814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0116e6b5-d804-4c43-a6f6-931f53fe446c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4v2p\" (UniqueName: \"kubernetes.io/projected/41e492fc-181e-4715-8fc9-26e02d17b789-kube-api-access-l4v2p\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.828990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829015 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thqj\" (UniqueName: \"kubernetes.io/projected/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-kube-api-access-9thqj\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzzvr\" (UniqueName: \"kubernetes.io/projected/11b7be65-4a8b-44e0-8177-b8b117407273-kube-api-access-nzzvr\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-trusted-ca\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.829125 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11b7be65-4a8b-44e0-8177-b8b117407273-cert\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scx7z\" (UniqueName: \"kubernetes.io/projected/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-kube-api-access-scx7z\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-tmp-dir\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-stats-auth\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829314 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-snapshots\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-default-certificate\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-serving-cert\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d0f68f-8bae-4925-afb4-bc71338d6599-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829414 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829428 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829432 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7xxb\" (UniqueName: \"kubernetes.io/projected/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-kube-api-access-z7xxb\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829453 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.829496 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829483 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.329466769 +0000 UTC m=+40.592485416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829487 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829404 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-config\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829526 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829728 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.329712507 +0000 UTC m=+40.592731155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829817 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.829871 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.329858085 +0000 UTC m=+40.592876732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnvfr\" (UniqueName: \"kubernetes.io/projected/8fa1148f-b2d8-4390-b715-08a448460d22-kube-api-access-cnvfr\") pod \"network-check-source-989f8b8dc-c46mt\" (UID: \"8fa1148f-b2d8-4390-b715-08a448460d22\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.829956 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-tmp-dir\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6d0f68f-8bae-4925-afb4-bc71338d6599-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w64lg\" (UniqueName: \"kubernetes.io/projected/d6d0f68f-8bae-4925-afb4-bc71338d6599-kube-api-access-w64lg\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830057 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-config-volume\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.830267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82e36f17-b33c-406c-9648-b0078e7c4184-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830341 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a33585d-426f-43fc-98c8-2fd6a1e00fef-trusted-ca\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830545 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a33585d-426f-43fc-98c8-2fd6a1e00fef-serving-cert\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgc5c\" (UniqueName: \"kubernetes.io/projected/0a33585d-426f-43fc-98c8-2fd6a1e00fef-kube-api-access-dgc5c\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-tmp\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830693 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830712 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-config-volume\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.830953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6d0f68f-8bae-4925-afb4-bc71338d6599-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.831662 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.830985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.831662 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.831043 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82e36f17-b33c-406c-9648-b0078e7c4184-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.831662 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.831189 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:34.831662 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.831247 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.331232048 +0000 UTC m=+40.594250714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:34.831662 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.831304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-tmp\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.831920 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.831879 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.832788 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.832765 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-serving-cert\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.832888 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.832815 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d0f68f-8bae-4925-afb4-bc71338d6599-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.833019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.832999 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.833173 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.833158 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a33585d-426f-43fc-98c8-2fd6a1e00fef-serving-cert\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.833436 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.833421 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.838535 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.838515 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll62z\" (UniqueName: \"kubernetes.io/projected/82e36f17-b33c-406c-9648-b0078e7c4184-kube-api-access-ll62z\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:34.839320 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.839283 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4b5r\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.841512 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.841488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:34.842192 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.842172 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thqj\" (UniqueName: \"kubernetes.io/projected/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-kube-api-access-9thqj\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:34.842329 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.842311 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scx7z\" (UniqueName: \"kubernetes.io/projected/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-kube-api-access-scx7z\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:34.845623 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.845601 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w64lg\" (UniqueName: \"kubernetes.io/projected/d6d0f68f-8bae-4925-afb4-bc71338d6599-kube-api-access-w64lg\") pod \"kube-storage-version-migrator-operator-846bf668f7-wbc58\" (UID: \"d6d0f68f-8bae-4925-afb4-bc71338d6599\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:34.846609 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.846590 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7xxb\" (UniqueName: \"kubernetes.io/projected/9ee637f4-2915-4fd8-9ce8-58d4629dbf65-kube-api-access-z7xxb\") pod \"insights-operator-5c847dcfc4-klzjd\" (UID: \"9ee637f4-2915-4fd8-9ce8-58d4629dbf65\") " pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:34.847458 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.847442 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgc5c\" (UniqueName: \"kubernetes.io/projected/0a33585d-426f-43fc-98c8-2fd6a1e00fef-kube-api-access-dgc5c\") pod \"console-operator-68b757fb67-7j7sj\" (UID: \"0a33585d-426f-43fc-98c8-2fd6a1e00fef\") " pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:34.907349 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.907321 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" Jul 08 15:51:34.924425 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.924406 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" Jul 08 15:51:34.931246 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0116e6b5-d804-4c43-a6f6-931f53fe446c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.931321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931251 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4v2p\" (UniqueName: \"kubernetes.io/projected/41e492fc-181e-4715-8fc9-26e02d17b789-kube-api-access-l4v2p\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.931321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzzvr\" (UniqueName: \"kubernetes.io/projected/11b7be65-4a8b-44e0-8177-b8b117407273-kube-api-access-nzzvr\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.931321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11b7be65-4a8b-44e0-8177-b8b117407273-cert\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.931321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.931510 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.931421 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.431403638 +0000 UTC m=+40.694422308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:34.931510 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-stats-auth\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.931648 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-default-certificate\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.931648 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnvfr\" (UniqueName: \"kubernetes.io/projected/8fa1148f-b2d8-4390-b715-08a448460d22-kube-api-access-cnvfr\") pod \"network-check-source-989f8b8dc-c46mt\" (UID: \"8fa1148f-b2d8-4390-b715-08a448460d22\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" Jul 08 15:51:34.931747 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.931747 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.931838 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.931770 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:34.931838 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.931821 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:34.931934 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.931853 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.43183555 +0000 UTC m=+40.694854214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:34.931934 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:34.931879 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:51:35.4318646 +0000 UTC m=+40.694883248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:34.932048 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.931989 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0116e6b5-d804-4c43-a6f6-931f53fe446c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:34.933915 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.933896 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-stats-auth\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.934060 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.934038 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11b7be65-4a8b-44e0-8177-b8b117407273-cert\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.934129 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.934113 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-default-certificate\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.942840 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.942817 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4v2p\" (UniqueName: \"kubernetes.io/projected/41e492fc-181e-4715-8fc9-26e02d17b789-kube-api-access-l4v2p\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:34.945126 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.945103 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnvfr\" (UniqueName: \"kubernetes.io/projected/8fa1148f-b2d8-4390-b715-08a448460d22-kube-api-access-cnvfr\") pod \"network-check-source-989f8b8dc-c46mt\" (UID: \"8fa1148f-b2d8-4390-b715-08a448460d22\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" Jul 08 15:51:34.947362 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.947340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzzvr\" (UniqueName: \"kubernetes.io/projected/11b7be65-4a8b-44e0-8177-b8b117407273-kube-api-access-nzzvr\") pod \"ingress-canary-8ppj7\" (UID: \"11b7be65-4a8b-44e0-8177-b8b117407273\") " pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:34.958747 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:34.958687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" Jul 08 15:51:35.016539 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.016346 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:35.030941 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.030804 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" Jul 08 15:51:35.048634 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.047248 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8ppj7" Jul 08 15:51:35.053828 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.053738 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" Jul 08 15:51:35.180057 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.179920 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw"] Jul 08 15:51:35.182765 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.181521 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8"] Jul 08 15:51:35.187733 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.187711 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58"] Jul 08 15:51:35.195643 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:51:35.194046 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09287bf3_d83b_445e_a731_2e541313c740.slice/crio-eda0778d504f61edd36daf063879bda1e76092a239b906dcaaa26730ec5d6294 WatchSource:0}: Error finding container eda0778d504f61edd36daf063879bda1e76092a239b906dcaaa26730ec5d6294: Status 404 returned error can't find the container with id eda0778d504f61edd36daf063879bda1e76092a239b906dcaaa26730ec5d6294 Jul 08 15:51:35.265379 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.265154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:51:35.265769 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.265275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:51:35.266141 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.265348 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:51:35.268132 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.267980 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 15:51:35.268132 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.267999 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-24mlw\"" Jul 08 15:51:35.268132 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.268118 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 15:51:35.268280 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.268254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8k7f\"" Jul 08 15:51:35.268982 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.268933 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7j7sj"] Jul 08 15:51:35.271412 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:51:35.271391 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a33585d_426f_43fc_98c8_2fd6a1e00fef.slice/crio-19c9b94935b96e930af3895bab61967d24f7324a6bb37aade634caff388e30d5 WatchSource:0}: Error finding container 19c9b94935b96e930af3895bab61967d24f7324a6bb37aade634caff388e30d5: Status 404 returned error can't find the container with id 19c9b94935b96e930af3895bab61967d24f7324a6bb37aade634caff388e30d5 Jul 08 15:51:35.280620 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.280600 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-klzjd"] Jul 08 15:51:35.283410 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:51:35.283391 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee637f4_2915_4fd8_9ce8_58d4629dbf65.slice/crio-c885517e71b35c61e2fa214db160abf5cb3557c463db55fe88621191c917cabb WatchSource:0}: Error finding container c885517e71b35c61e2fa214db160abf5cb3557c463db55fe88621191c917cabb: Status 404 returned error can't find the container with id c885517e71b35c61e2fa214db160abf5cb3557c463db55fe88621191c917cabb Jul 08 15:51:35.299079 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.299002 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppj7"] Jul 08 15:51:35.302747 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:51:35.302727 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b7be65_4a8b_44e0_8177_b8b117407273.slice/crio-b12fd0228f952b9de6243529cb7f1adb94e0b77ed25ac7236237f431c207b23b WatchSource:0}: Error finding container b12fd0228f952b9de6243529cb7f1adb94e0b77ed25ac7236237f431c207b23b: Status 404 returned error can't find the container with id b12fd0228f952b9de6243529cb7f1adb94e0b77ed25ac7236237f431c207b23b Jul 08 15:51:35.303163 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.303145 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt"] Jul 08 15:51:35.306168 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:51:35.306151 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa1148f_b2d8_4390_b715_08a448460d22.slice/crio-3d64eec8398846a7efc5e2597c75e0f2fc81d3954beedb418d8a60c97b139ef9 WatchSource:0}: Error finding container 3d64eec8398846a7efc5e2597c75e0f2fc81d3954beedb418d8a60c97b139ef9: Status 404 returned error can't find the container with id 3d64eec8398846a7efc5e2597c75e0f2fc81d3954beedb418d8a60c97b139ef9 Jul 08 15:51:35.335244 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.335221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:35.335323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.335278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:35.335323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.335302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:35.335422 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.335351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:35.335422 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335357 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:35.335422 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335373 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335423 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.335402804 +0000 UTC m=+41.598421475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335446 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335462 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335446 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335502 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.335483172 +0000 UTC m=+41.598501834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335520 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.335510334 +0000 UTC m=+41.598528986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:35.335557 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.335535 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.335525976 +0000 UTC m=+41.598544624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:35.435714 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.435690 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:35.435802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.435757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:35.435802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.435776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:35.435908 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.435831 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.435817473 +0000 UTC m=+41.698836120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:35.435908 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.435845 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:35.435908 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.435877 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.435866416 +0000 UTC m=+41.698885063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:35.436032 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.435908 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:35.436032 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:35.435956 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:36.435940198 +0000 UTC m=+41.698958860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:35.454233 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.454215 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" event={"ID":"9ee637f4-2915-4fd8-9ce8-58d4629dbf65","Type":"ContainerStarted","Data":"c885517e71b35c61e2fa214db160abf5cb3557c463db55fe88621191c917cabb"} Jul 08 15:51:35.455302 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.455285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8ppj7" event={"ID":"11b7be65-4a8b-44e0-8177-b8b117407273","Type":"ContainerStarted","Data":"b12fd0228f952b9de6243529cb7f1adb94e0b77ed25ac7236237f431c207b23b"} Jul 08 15:51:35.456261 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.456240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" event={"ID":"8fa1148f-b2d8-4390-b715-08a448460d22","Type":"ContainerStarted","Data":"3d64eec8398846a7efc5e2597c75e0f2fc81d3954beedb418d8a60c97b139ef9"} Jul 08 15:51:35.457256 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.457236 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" event={"ID":"0a33585d-426f-43fc-98c8-2fd6a1e00fef","Type":"ContainerStarted","Data":"19c9b94935b96e930af3895bab61967d24f7324a6bb37aade634caff388e30d5"} Jul 08 15:51:35.458303 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.458275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" event={"ID":"09287bf3-d83b-445e-a731-2e541313c740","Type":"ContainerStarted","Data":"eda0778d504f61edd36daf063879bda1e76092a239b906dcaaa26730ec5d6294"} Jul 08 15:51:35.459393 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.459374 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" event={"ID":"2ad66488-6def-4a0d-8f34-3a8a499f0800","Type":"ContainerStarted","Data":"f25ea1ca3f5000c2b2274eeeaf7ff95e7b272d9e16aa4555fd0550e80fe24637"} Jul 08 15:51:35.460396 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:35.460376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" event={"ID":"d6d0f68f-8bae-4925-afb4-bc71338d6599","Type":"ContainerStarted","Data":"50b2d819f2e9010ff29ab117a450f50aa1ccda581d0bc74804345f2a2a26a591"} Jul 08 15:51:36.345724 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.345686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:36.345895 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.345781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:36.345895 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.345819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:36.345895 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.345882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:36.346079 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346053 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:36.346133 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346113 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.346093641 +0000 UTC m=+43.609112303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:36.346560 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346539 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:36.346560 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346562 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:36.346724 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346638 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.346621501 +0000 UTC m=+43.609640162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:36.346724 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346704 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:36.346830 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346734 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.346723732 +0000 UTC m=+43.609742395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:36.346830 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346792 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:36.346830 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.346823 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.346812914 +0000 UTC m=+43.609831575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:36.447185 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.447153 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:36.447381 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.447262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:36.447381 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:36.447292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:36.447499 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.447482 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:36.447554 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.447537 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.447518995 +0000 UTC m=+43.710537643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:36.447952 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.447934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.44791842 +0000 UTC m=+43.710937087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:36.448063 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.448002 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:36.448063 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:36.448028 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:38.448019056 +0000 UTC m=+43.711037716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:38.367264 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.367233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.367292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.367316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367384 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367390 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367403 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367393 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367449 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.367434381 +0000 UTC m=+47.630453051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367463 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.367456673 +0000 UTC m=+47.630475320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.367491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367536 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.367513154 +0000 UTC m=+47.630531811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367546 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:38.367763 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.367596 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.367588443 +0000 UTC m=+47.630607090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:38.468247 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.468218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:38.468395 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.468304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:38.468395 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.468373 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:38.468559 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.468390 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.468369685 +0000 UTC m=+47.731388355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:38.468559 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:38.468420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:38.468559 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.468497 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.468482105 +0000 UTC m=+47.731500758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:38.468559 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.468513 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:38.468559 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:38.468550 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:51:42.468541642 +0000 UTC m=+47.731560288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.401371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.401483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.401520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401546 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401589 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401651 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.401635387 +0000 UTC m=+55.664654052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401659 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.401688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401721 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401726 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.401710337 +0000 UTC m=+55.664729001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401755 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401762 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.401750561 +0000 UTC m=+55.664769208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:42.401794 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.401782 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.401773415 +0000 UTC m=+55.664792098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:42.502957 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.502927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:42.503100 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.503009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:42.503100 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:42.503027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:42.503168 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.503110 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.503084874 +0000 UTC m=+55.766103529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:42.503168 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.503139 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:42.503168 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.503157 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:42.503258 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.503180 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.503170182 +0000 UTC m=+55.766188852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:42.503258 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:42.503204 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:50.503191694 +0000 UTC m=+55.766210365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:43.490910 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.490419 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" event={"ID":"9ee637f4-2915-4fd8-9ce8-58d4629dbf65","Type":"ContainerStarted","Data":"321775c88dd8ae90830d0f4e90798fe33e630a3d61a1b1f9aff3a099216f49e7"} Jul 08 15:51:43.492935 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.492903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8ppj7" event={"ID":"11b7be65-4a8b-44e0-8177-b8b117407273","Type":"ContainerStarted","Data":"609a7b60281974792e46a707f2a0b7519de5904acd478e18b3316e77448c1071"} Jul 08 15:51:43.494804 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.494777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" event={"ID":"8fa1148f-b2d8-4390-b715-08a448460d22","Type":"ContainerStarted","Data":"d515aff837c23dff09af224053ff89dfc88c6576fffc1ead02e39dd64d7b8a46"} Jul 08 15:51:43.496785 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.496769 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/0.log" Jul 08 15:51:43.496878 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.496802 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a33585d-426f-43fc-98c8-2fd6a1e00fef" containerID="f97d2e7a84006c6aa27a4b9da02962b5c3d3b98763af7f2c38d461113e54fc20" exitCode=255 Jul 08 15:51:43.496945 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.496882 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" event={"ID":"0a33585d-426f-43fc-98c8-2fd6a1e00fef","Type":"ContainerDied","Data":"f97d2e7a84006c6aa27a4b9da02962b5c3d3b98763af7f2c38d461113e54fc20"} Jul 08 15:51:43.497053 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.497038 2566 scope.go:117] "RemoveContainer" containerID="f97d2e7a84006c6aa27a4b9da02962b5c3d3b98763af7f2c38d461113e54fc20" Jul 08 15:51:43.498892 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.498598 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" event={"ID":"09287bf3-d83b-445e-a731-2e541313c740","Type":"ContainerStarted","Data":"ee67d2359de81e2dac00cc7b9a30ba6e8adb63f2b1fdde8f5fa13c4e511eac1d"} Jul 08 15:51:43.500613 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.500545 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" event={"ID":"2ad66488-6def-4a0d-8f34-3a8a499f0800","Type":"ContainerStarted","Data":"33080194dfc42f2c1f812b14e84dc660004a824385c3bf38e30ab7e50dee3b90"} Jul 08 15:51:43.502015 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.501995 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" event={"ID":"d6d0f68f-8bae-4925-afb4-bc71338d6599","Type":"ContainerStarted","Data":"f4bb95f376743f9d4304179d955fd6fd1641a2f17471c7fa3dd07d47e81c8c6c"} Jul 08 15:51:43.510268 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.510221 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" podStartSLOduration=14.760504981 podStartE2EDuration="22.510104144s" totalImagesPullingTime="7.749599163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.508689858 +0000 UTC m=+48.771708520" watchObservedRunningTime="2026-07-08 15:51:43.510104144 +0000 UTC m=+48.773122811" Jul 08 15:51:43.524898 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.524851 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" podStartSLOduration=14.703305809 podStartE2EDuration="22.524838486s" totalImagesPullingTime="7.821532677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.523590023 +0000 UTC m=+48.786608692" watchObservedRunningTime="2026-07-08 15:51:43.524838486 +0000 UTC m=+48.787857154" Jul 08 15:51:43.562129 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.562081 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" podStartSLOduration=14.740592301 podStartE2EDuration="22.562062599s" totalImagesPullingTime="7.821470298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.561052887 +0000 UTC m=+48.824071577" watchObservedRunningTime="2026-07-08 15:51:43.562062599 +0000 UTC m=+48.825081273" Jul 08 15:51:43.592825 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.592754 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cmwrw" podStartSLOduration=14.807659558 podStartE2EDuration="22.592736219s" totalImagesPullingTime="7.785076661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.576194144 +0000 UTC m=+48.839212815" watchObservedRunningTime="2026-07-08 15:51:43.592736219 +0000 UTC m=+48.855754928" Jul 08 15:51:43.593281 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.593249 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8ppj7" podStartSLOduration=1.8551496269999999 podStartE2EDuration="9.593236702s" totalImagesPullingTime="7.738087075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.592486248 +0000 UTC m=+48.855504918" watchObservedRunningTime="2026-07-08 15:51:43.593236702 +0000 UTC m=+48.856255369" Jul 08 15:51:43.620622 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:43.620582 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c46mt" podStartSLOduration=14.892273788 podStartE2EDuration="22.620550898s" totalImagesPullingTime="7.72827711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:43.619945639 +0000 UTC m=+48.882964309" watchObservedRunningTime="2026-07-08 15:51:43.620550898 +0000 UTC m=+48.883569567" Jul 08 15:51:44.508703 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.508661 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 15:51:44.509203 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.509184 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/0.log" Jul 08 15:51:44.509271 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.509222 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a33585d-426f-43fc-98c8-2fd6a1e00fef" containerID="ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a" exitCode=255 Jul 08 15:51:44.509993 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.509975 2566 scope.go:117] "RemoveContainer" containerID="ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a" Jul 08 15:51:44.510152 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:44.510125 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7j7sj_openshift-console-operator(0a33585d-426f-43fc-98c8-2fd6a1e00fef)\"" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" podUID="0a33585d-426f-43fc-98c8-2fd6a1e00fef" Jul 08 15:51:44.510222 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.510170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" event={"ID":"0a33585d-426f-43fc-98c8-2fd6a1e00fef","Type":"ContainerDied","Data":"ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a"} Jul 08 15:51:44.510275 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:44.510247 2566 scope.go:117] "RemoveContainer" containerID="f97d2e7a84006c6aa27a4b9da02962b5c3d3b98763af7f2c38d461113e54fc20" Jul 08 15:51:45.017423 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.017393 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:45.017603 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.017438 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:45.230832 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.230804 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7w2mq"] Jul 08 15:51:45.244850 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.244822 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.250426 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.250405 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7w2mq"] Jul 08 15:51:45.251168 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.251140 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 15:51:45.252144 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.252115 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-96wh4\"" Jul 08 15:51:45.253752 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.253735 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 15:51:45.334702 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.334637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.334702 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.334665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6142acdd-6d9e-4979-aa6e-85ad14509660-crio-socket\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.334702 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.334692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6142acdd-6d9e-4979-aa6e-85ad14509660-data-volume\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.334949 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.334752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.334949 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.334830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr7sf\" (UniqueName: \"kubernetes.io/projected/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-api-access-wr7sf\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436034 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436143 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6142acdd-6d9e-4979-aa6e-85ad14509660-crio-socket\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436143 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6142acdd-6d9e-4979-aa6e-85ad14509660-data-volume\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436143 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436143 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.436131 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 15:51:45.436305 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436145 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6142acdd-6d9e-4979-aa6e-85ad14509660-crio-socket\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436305 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr7sf\" (UniqueName: \"kubernetes.io/projected/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-api-access-wr7sf\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.436305 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.436189 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:45.936167281 +0000 UTC m=+51.199185941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : configmap "kube-rbac-proxy" not found Jul 08 15:51:45.436305 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.436222 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:51:45.436305 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.436257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:45.936246124 +0000 UTC m=+51.199264806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:51:45.436500 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.436401 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6142acdd-6d9e-4979-aa6e-85ad14509660-data-volume\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.446047 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.446027 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr7sf\" (UniqueName: \"kubernetes.io/projected/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-api-access-wr7sf\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.513927 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.513909 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 15:51:45.514216 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.514172 2566 scope.go:117] "RemoveContainer" containerID="ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a" Jul 08 15:51:45.514318 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.514300 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7j7sj_openshift-console-operator(0a33585d-426f-43fc-98c8-2fd6a1e00fef)\"" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" podUID="0a33585d-426f-43fc-98c8-2fd6a1e00fef" Jul 08 15:51:45.940742 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.940707 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.940942 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:45.940827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:45.940942 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.940867 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:51:45.940942 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.940882 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 15:51:45.940942 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.940938 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:46.94092521 +0000 UTC m=+52.203943858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : configmap "kube-rbac-proxy" not found Jul 08 15:51:45.941099 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:45.940952 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:46.940945609 +0000 UTC m=+52.203964256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:51:46.517235 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:46.517204 2566 scope.go:117] "RemoveContainer" containerID="ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a" Jul 08 15:51:46.517643 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:46.517369 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7j7sj_openshift-console-operator(0a33585d-426f-43fc-98c8-2fd6a1e00fef)\"" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" podUID="0a33585d-426f-43fc-98c8-2fd6a1e00fef" Jul 08 15:51:46.949692 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:46.949661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:46.949854 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:46.949724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:46.949854 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:46.949779 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 15:51:46.949854 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:46.949809 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:51:46.949854 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:46.949841 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:48.949825278 +0000 UTC m=+54.212843924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : configmap "kube-rbac-proxy" not found Jul 08 15:51:46.949987 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:46.949858 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:48.949847587 +0000 UTC m=+54.212866235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:51:47.611520 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:47.611495 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h6n7j_8ead0a15-65c5-47a6-b1b6-10bdf1b92031/dns-node-resolver/0.log" Jul 08 15:51:48.809098 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:48.809070 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-drs65_91a472ac-8359-42de-8fb2-3fae3c6117c0/node-ca/0.log" Jul 08 15:51:48.965128 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:48.965100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:48.965256 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:48.965165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:48.965256 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:48.965223 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 15:51:48.965322 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:48.965275 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:51:48.965322 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:48.965300 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:52.965281908 +0000 UTC m=+58.228300564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : configmap "kube-rbac-proxy" not found Jul 08 15:51:48.965322 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:48.965319 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:51:52.96530963 +0000 UTC m=+58.228328278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:51:49.209769 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:49.209744 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8ppj7_11b7be65-4a8b-44e0-8177-b8b117407273/serve-healthcheck-canary/0.log" Jul 08 15:51:50.010986 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.010960 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wbc58_d6d0f68f-8bae-4925-afb4-bc71338d6599/kube-storage-version-migrator-operator/0.log" Jul 08 15:51:50.477152 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.477121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:51:50.477319 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.477159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:51:50.477319 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.477208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:51:50.477319 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.477269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:51:50.477319 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477282 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477344 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls podName:7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.477327173 +0000 UTC m=+71.740345832 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls") pod "dns-default-c79v7" (UID: "7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9") : secret "dns-default-metrics-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477375 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477405 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477415 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d648f84bb-dx2kk: secret "image-registry-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477433 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls podName:82e36f17-b33c-406c-9648-b0078e7c4184 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.477420963 +0000 UTC m=+71.740439629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lf9rm" (UID: "82e36f17-b33c-406c-9648-b0078e7c4184") : secret "cluster-monitoring-operator-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477442 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477449 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls podName:244c193e-795a-4ed3-9c97-6a0007acec2e nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.477438492 +0000 UTC m=+71.740457180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls") pod "image-registry-6d648f84bb-dx2kk" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e") : secret "image-registry-tls" not found Jul 08 15:51:50.477527 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.477494 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls podName:fb2cea25-bd8d-479a-8fcb-0fc980fee58a nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.477477035 +0000 UTC m=+71.740495696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bjxzl" (UID: "fb2cea25-bd8d-479a-8fcb-0fc980fee58a") : secret "samples-operator-tls" not found Jul 08 15:51:50.578237 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.578208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:50.578394 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.578243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:51:50.578394 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:50.578348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:51:50.578394 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.578373 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 15:51:50.578589 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.578425 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert podName:0116e6b5-d804-4c43-a6f6-931f53fe446c nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.578409526 +0000 UTC m=+71.841428173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-g6nhn" (UID: "0116e6b5-d804-4c43-a6f6-931f53fe446c") : secret "networking-console-plugin-cert" not found Jul 08 15:51:50.578589 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.578429 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 15:51:50.578589 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.578517 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.578496197 +0000 UTC m=+71.841514865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : configmap references non-existent config key: service-ca.crt Jul 08 15:51:50.578589 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:50.578561 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs podName:41e492fc-181e-4715-8fc9-26e02d17b789 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:06.57854443 +0000 UTC m=+71.841563079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs") pod "router-default-bdcbbdb9-r5lfj" (UID: "41e492fc-181e-4715-8fc9-26e02d17b789") : secret "router-metrics-certs-default" not found Jul 08 15:51:53.000196 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:53.000160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:53.000611 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:53.000225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:51:53.000611 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:53.000295 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 15:51:53.000611 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:53.000317 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:51:53.000611 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:53.000364 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:01.000351123 +0000 UTC m=+66.263369789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:51:53.000611 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:51:53.000377 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:01.00037088 +0000 UTC m=+66.263389527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : configmap "kube-rbac-proxy" not found Jul 08 15:51:59.264079 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.264051 2566 scope.go:117] "RemoveContainer" containerID="ac69d16c27e899a5c763f656747b41bb2b628c2a4529cb71b4771f41e9646f1a" Jul 08 15:51:59.442616 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.442588 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5np9" Jul 08 15:51:59.556681 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.556622 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 15:51:59.556681 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.556668 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" event={"ID":"0a33585d-426f-43fc-98c8-2fd6a1e00fef","Type":"ContainerStarted","Data":"4f96a049086d1052c8e56b5774a88eb96d2ad46d6110eba77ae364f6374fbb36"} Jul 08 15:51:59.556969 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.556949 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:51:59.579741 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.579697 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" podStartSLOduration=30.81725962 podStartE2EDuration="38.579683304s" totalImagesPullingTime="7.762423684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:51:59.578693913 +0000 UTC m=+64.841712584" watchObservedRunningTime="2026-07-08 15:51:59.579683304 +0000 UTC m=+64.842701987" Jul 08 15:51:59.773012 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:51:59.772984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-7j7sj" Jul 08 15:52:00.967213 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:00.967184 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:52:00.967608 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:00.967250 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:52:00.969688 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:00.969668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98sr8\" (UniqueName: \"kubernetes.io/projected/2c3f5924-af89-4e1a-a4c3-3b75d64b22fb-kube-api-access-98sr8\") pod \"network-check-target-vlnjs\" (UID: \"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb\") " pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:52:00.969980 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:00.969965 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 15:52:00.979646 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:00.979621 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48789b43-33de-4682-aeb5-06bbde0259e9-original-pull-secret\") pod \"global-pull-secret-syncer-tczlz\" (UID: \"48789b43-33de-4682-aeb5-06bbde0259e9\") " pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:52:01.068354 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.068319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:52:01.068514 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.068374 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:01.068514 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.068453 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:01.068657 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:01.068535 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 15:52:01.068657 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:01.068604 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls podName:6142acdd-6d9e-4979-aa6e-85ad14509660 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:17.068591582 +0000 UTC m=+82.331610229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7w2mq" (UID: "6142acdd-6d9e-4979-aa6e-85ad14509660") : secret "insights-runtime-extractor-tls" not found Jul 08 15:52:01.068885 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.068864 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6142acdd-6d9e-4979-aa6e-85ad14509660-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:01.071181 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.071165 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 15:52:01.080936 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.080908 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac03da3-cc5b-498c-b834-11719d2f4a21-metrics-certs\") pod \"network-metrics-daemon-772cg\" (UID: \"0ac03da3-cc5b-498c-b834-11719d2f4a21\") " pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:52:01.084881 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.084861 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tczlz" Jul 08 15:52:01.094667 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.094648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8k7f\"" Jul 08 15:52:01.101965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.101936 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:52:01.220427 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.220340 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tczlz"] Jul 08 15:52:01.222854 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:01.222824 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48789b43_33de_4682_aeb5_06bbde0259e9.slice/crio-09aaf7bf6be6696906348e1928129f5e6c65f69dfae5710e4615f0facb2f25f7 WatchSource:0}: Error finding container 09aaf7bf6be6696906348e1928129f5e6c65f69dfae5710e4615f0facb2f25f7: Status 404 returned error can't find the container with id 09aaf7bf6be6696906348e1928129f5e6c65f69dfae5710e4615f0facb2f25f7 Jul 08 15:52:01.239669 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.239538 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlnjs"] Jul 08 15:52:01.241814 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:01.241792 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3f5924_af89_4e1a_a4c3_3b75d64b22fb.slice/crio-132d3c5ea0775e9f4020a5110ef6cf8d20a0274591bd7e946ae5c9982e4fb82f WatchSource:0}: Error finding container 132d3c5ea0775e9f4020a5110ef6cf8d20a0274591bd7e946ae5c9982e4fb82f: Status 404 returned error can't find the container with id 132d3c5ea0775e9f4020a5110ef6cf8d20a0274591bd7e946ae5c9982e4fb82f Jul 08 15:52:01.380231 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.380211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-24mlw\"" Jul 08 15:52:01.388957 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.388940 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-772cg" Jul 08 15:52:01.522254 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.522230 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-772cg"] Jul 08 15:52:01.524911 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:01.524887 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac03da3_cc5b_498c_b834_11719d2f4a21.slice/crio-435c119664c9f984546108201d0e78521f159681c7139d942b8fa1548806265d WatchSource:0}: Error finding container 435c119664c9f984546108201d0e78521f159681c7139d942b8fa1548806265d: Status 404 returned error can't find the container with id 435c119664c9f984546108201d0e78521f159681c7139d942b8fa1548806265d Jul 08 15:52:01.563978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.563955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tczlz" event={"ID":"48789b43-33de-4682-aeb5-06bbde0259e9","Type":"ContainerStarted","Data":"09aaf7bf6be6696906348e1928129f5e6c65f69dfae5710e4615f0facb2f25f7"} Jul 08 15:52:01.565422 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.565390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlnjs" event={"ID":"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb","Type":"ContainerStarted","Data":"f2a68d7bb94bd7ba0cdc30e669a1e55c165d48ced36f549d214779fe99317e02"} Jul 08 15:52:01.565525 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.565435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlnjs" event={"ID":"2c3f5924-af89-4e1a-a4c3-3b75d64b22fb","Type":"ContainerStarted","Data":"132d3c5ea0775e9f4020a5110ef6cf8d20a0274591bd7e946ae5c9982e4fb82f"} Jul 08 15:52:01.565612 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.565596 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:52:01.566658 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.566636 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-772cg" event={"ID":"0ac03da3-cc5b-498c-b834-11719d2f4a21","Type":"ContainerStarted","Data":"435c119664c9f984546108201d0e78521f159681c7139d942b8fa1548806265d"} Jul 08 15:52:01.583637 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:01.583593 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vlnjs" podStartSLOduration=65.583558318 podStartE2EDuration="1m5.583558318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:01.582545518 +0000 UTC m=+66.845564187" watchObservedRunningTime="2026-07-08 15:52:01.583558318 +0000 UTC m=+66.846576987" Jul 08 15:52:03.575110 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:03.575069 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-772cg" event={"ID":"0ac03da3-cc5b-498c-b834-11719d2f4a21","Type":"ContainerStarted","Data":"d646565b5d17b2c6ea82be33d1f2f6135849db8d44d291dd45017e4aa3898264"} Jul 08 15:52:03.575560 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:03.575137 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-772cg" event={"ID":"0ac03da3-cc5b-498c-b834-11719d2f4a21","Type":"ContainerStarted","Data":"76f923c83972d7e5adc0c73be6dc8f65f9b024e45a615182d00f7ac7136cafbe"} Jul 08 15:52:03.592259 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:03.592209 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-772cg" podStartSLOduration=66.259503265 podStartE2EDuration="1m7.592194262s" totalImagesPullingTime="1.332690997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:03.590723868 +0000 UTC m=+68.853742538" watchObservedRunningTime="2026-07-08 15:52:03.592194262 +0000 UTC m=+68.855212930" Jul 08 15:52:06.514802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.514763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:52:06.515249 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.514869 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:06.515249 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.514921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:52:06.515249 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.515067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:52:06.517698 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.517668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"image-registry-6d648f84bb-dx2kk\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:06.517821 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.517794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82e36f17-b33c-406c-9648-b0078e7c4184-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lf9rm\" (UID: \"82e36f17-b33c-406c-9648-b0078e7c4184\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:52:06.517887 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.517837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2cea25-bd8d-479a-8fcb-0fc980fee58a-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bjxzl\" (UID: \"fb2cea25-bd8d-479a-8fcb-0fc980fee58a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:52:06.518077 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.518057 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9-metrics-tls\") pod \"dns-default-c79v7\" (UID: \"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9\") " pod="openshift-dns/dns-default-c79v7" Jul 08 15:52:06.616051 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.616025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:06.616172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.616057 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:52:06.616172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.616121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:06.616784 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.616764 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41e492fc-181e-4715-8fc9-26e02d17b789-service-ca-bundle\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:06.618450 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.618431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41e492fc-181e-4715-8fc9-26e02d17b789-metrics-certs\") pod \"router-default-bdcbbdb9-r5lfj\" (UID: \"41e492fc-181e-4715-8fc9-26e02d17b789\") " pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:06.618548 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.618493 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0116e6b5-d804-4c43-a6f6-931f53fe446c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-g6nhn\" (UID: \"0116e6b5-d804-4c43-a6f6-931f53fe446c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:52:06.753338 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.753310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rpkvs\"" Jul 08 15:52:06.762309 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.762288 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:06.781237 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.781218 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mjhcc\"" Jul 08 15:52:06.789133 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.789112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c79v7" Jul 08 15:52:06.791335 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.791310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9cfl2\"" Jul 08 15:52:06.800110 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.800089 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" Jul 08 15:52:06.806357 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.806340 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6j6qt\"" Jul 08 15:52:06.817105 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.816918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" Jul 08 15:52:06.867397 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.867369 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pgxtn\"" Jul 08 15:52:06.875309 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.875286 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:06.879199 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.879165 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bcnh2\"" Jul 08 15:52:06.888031 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.888010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" Jul 08 15:52:06.899737 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:06.899646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:52:08.190308 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:08.190264 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244c193e_795a_4ed3_9c97_6a0007acec2e.slice/crio-5b1175a03cdf35299b024f3cddcb03dbf20721bf9e5756b685e0dbaa362d304a WatchSource:0}: Error finding container 5b1175a03cdf35299b024f3cddcb03dbf20721bf9e5756b685e0dbaa362d304a: Status 404 returned error can't find the container with id 5b1175a03cdf35299b024f3cddcb03dbf20721bf9e5756b685e0dbaa362d304a Jul 08 15:52:08.428620 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.428262 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl"] Jul 08 15:52:08.471726 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.471445 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm"] Jul 08 15:52:08.471726 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.471634 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c79v7"] Jul 08 15:52:08.474817 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:08.474780 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e36f17_b33c_406c_9648_b0078e7c4184.slice/crio-a48ad7fd1d5b6035f5c01cb8a63532b542f2a38f5ec4cf32ca5f4028912d7595 WatchSource:0}: Error finding container a48ad7fd1d5b6035f5c01cb8a63532b542f2a38f5ec4cf32ca5f4028912d7595: Status 404 returned error can't find the container with id a48ad7fd1d5b6035f5c01cb8a63532b542f2a38f5ec4cf32ca5f4028912d7595 Jul 08 15:52:08.478990 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:08.478966 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5dd1c8_44a9_4fcd_88eb_e3c32674ddf9.slice/crio-43b267928498fd043d573f1201ecd50c4a85c089e723664f63e88cab30bf3589 WatchSource:0}: Error finding container 43b267928498fd043d573f1201ecd50c4a85c089e723664f63e88cab30bf3589: Status 404 returned error can't find the container with id 43b267928498fd043d573f1201ecd50c4a85c089e723664f63e88cab30bf3589 Jul 08 15:52:08.511796 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.511758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn"] Jul 08 15:52:08.514190 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.514141 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bdcbbdb9-r5lfj"] Jul 08 15:52:08.594195 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.594161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tczlz" event={"ID":"48789b43-33de-4682-aeb5-06bbde0259e9","Type":"ContainerStarted","Data":"7327c6c977f99e2ab33c83d9fae163491986a0f3ec212ff31ef3b57de466519f"} Jul 08 15:52:08.595869 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.595840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" event={"ID":"41e492fc-181e-4715-8fc9-26e02d17b789","Type":"ContainerStarted","Data":"41091e9be859b60d707786ce95ddbc4b6d5596f3d496ee59fb0f8edc37bd60ad"} Jul 08 15:52:08.597383 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.597183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c79v7" event={"ID":"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9","Type":"ContainerStarted","Data":"43b267928498fd043d573f1201ecd50c4a85c089e723664f63e88cab30bf3589"} Jul 08 15:52:08.601243 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.601192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" event={"ID":"244c193e-795a-4ed3-9c97-6a0007acec2e","Type":"ContainerStarted","Data":"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70"} Jul 08 15:52:08.601243 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.601240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" event={"ID":"244c193e-795a-4ed3-9c97-6a0007acec2e","Type":"ContainerStarted","Data":"5b1175a03cdf35299b024f3cddcb03dbf20721bf9e5756b685e0dbaa362d304a"} Jul 08 15:52:08.601409 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.601265 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:08.602730 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.602686 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" event={"ID":"fb2cea25-bd8d-479a-8fcb-0fc980fee58a","Type":"ContainerStarted","Data":"20d9cc6866b6fdf63b63e5aaf3b4a754f34c6a83721f04dd9fbe0f12a7020d83"} Jul 08 15:52:08.604899 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.604878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" event={"ID":"0116e6b5-d804-4c43-a6f6-931f53fe446c","Type":"ContainerStarted","Data":"fec1d69964788a88e9d80cbcb9ada924dda9db56410e5422be3aa23f7294100c"} Jul 08 15:52:08.606178 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.606158 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" event={"ID":"82e36f17-b33c-406c-9648-b0078e7c4184","Type":"ContainerStarted","Data":"a48ad7fd1d5b6035f5c01cb8a63532b542f2a38f5ec4cf32ca5f4028912d7595"} Jul 08 15:52:08.631172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.631110 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tczlz" podStartSLOduration=65.553606078 podStartE2EDuration="1m12.631088547s" totalImagesPullingTime="7.077482469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:08.62789199 +0000 UTC m=+73.890910660" watchObservedRunningTime="2026-07-08 15:52:08.631088547 +0000 UTC m=+73.894107218" Jul 08 15:52:08.656283 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:08.656237 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" podStartSLOduration=72.65622172 podStartE2EDuration="1m12.65622172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:08.654700299 +0000 UTC m=+73.917718992" watchObservedRunningTime="2026-07-08 15:52:08.65622172 +0000 UTC m=+73.919240388" Jul 08 15:52:09.613119 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:09.612930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" event={"ID":"41e492fc-181e-4715-8fc9-26e02d17b789","Type":"ContainerStarted","Data":"f3eea896e780a76387688dcb130749f0a71488d82c7cb96efa12e6f8cf59ff72"} Jul 08 15:52:09.636597 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:09.636525 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" podStartSLOduration=48.636504625 podStartE2EDuration="48.636504625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:09.634633575 +0000 UTC m=+74.897652245" watchObservedRunningTime="2026-07-08 15:52:09.636504625 +0000 UTC m=+74.899523296" Jul 08 15:52:09.876428 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:09.876352 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:09.879335 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:09.879311 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:10.617864 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:10.617833 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:10.618976 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:10.618955 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-bdcbbdb9-r5lfj" Jul 08 15:52:12.629586 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.629525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" event={"ID":"0116e6b5-d804-4c43-a6f6-931f53fe446c","Type":"ContainerStarted","Data":"d8b0dadd9621d0996226149bde85b184d1e2553c81f69dfa092b0fe735becc8e"} Jul 08 15:52:12.631117 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.631097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" event={"ID":"82e36f17-b33c-406c-9648-b0078e7c4184","Type":"ContainerStarted","Data":"c2f4ec99743f0f2b684500f904a438759e7e9b156d9586f7b8c956763e72f3c1"} Jul 08 15:52:12.632742 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.632722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c79v7" event={"ID":"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9","Type":"ContainerStarted","Data":"4eea650995522d389fed24cabeafaf0181f79275852a6997592cce352e149c14"} Jul 08 15:52:12.632819 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.632759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c79v7" event={"ID":"7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9","Type":"ContainerStarted","Data":"45adf327b9d1e18ad477d98887e3f59362758883eeff4fc82dbd990bf766e10c"} Jul 08 15:52:12.632819 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.632789 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c79v7" Jul 08 15:52:12.634369 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.634350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" event={"ID":"fb2cea25-bd8d-479a-8fcb-0fc980fee58a","Type":"ContainerStarted","Data":"b61fbc8083c8a20a5e0cf70baab6e756d9df2e2f0963d77720b4d497cc367f90"} Jul 08 15:52:12.634448 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.634385 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" event={"ID":"fb2cea25-bd8d-479a-8fcb-0fc980fee58a","Type":"ContainerStarted","Data":"4f6a236a5c84c77ab63b6294996f16243dbe3ace0b5365886b0125c06099937f"} Jul 08 15:52:12.646150 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.646112 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-g6nhn" podStartSLOduration=48.656503789 podStartE2EDuration="51.646098069s" totalImagesPullingTime="2.98959428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:12.645282234 +0000 UTC m=+77.908300902" watchObservedRunningTime="2026-07-08 15:52:12.646098069 +0000 UTC m=+77.909116738" Jul 08 15:52:12.665453 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.665416 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c79v7" podStartSLOduration=35.634095809 podStartE2EDuration="38.665403413s" totalImagesPullingTime="3.031307604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:12.664185879 +0000 UTC m=+77.927204569" watchObservedRunningTime="2026-07-08 15:52:12.665403413 +0000 UTC m=+77.928422082" Jul 08 15:52:12.680483 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.680446 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lf9rm" podStartSLOduration=48.595964065 podStartE2EDuration="51.68043352s" totalImagesPullingTime="3.084469455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:12.679969005 +0000 UTC m=+77.942987688" watchObservedRunningTime="2026-07-08 15:52:12.68043352 +0000 UTC m=+77.943452234" Jul 08 15:52:12.695995 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:12.695951 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bjxzl" podStartSLOduration=48.656869829 podStartE2EDuration="51.69593322s" totalImagesPullingTime="3.039063391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:12.695278042 +0000 UTC m=+77.958296711" watchObservedRunningTime="2026-07-08 15:52:12.69593322 +0000 UTC m=+77.958951893" Jul 08 15:52:17.105845 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.105715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:17.108071 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.108051 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6142acdd-6d9e-4979-aa6e-85ad14509660-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7w2mq\" (UID: \"6142acdd-6d9e-4979-aa6e-85ad14509660\") " pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:17.160724 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.160691 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5d754"] Jul 08 15:52:17.179725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.179702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5d754"] Jul 08 15:52:17.179822 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.179796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.182814 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.182793 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 15:52:17.182935 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.182815 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 15:52:17.182935 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.182830 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-n2n5z\"" Jul 08 15:52:17.182935 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.182815 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 15:52:17.206846 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.206828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f943243-e356-4d94-8635-709576aa00e6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.206951 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.206861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.206951 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.206894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.206951 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.206909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk2q2\" (UniqueName: \"kubernetes.io/projected/0f943243-e356-4d94-8635-709576aa00e6-kube-api-access-nk2q2\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.307353 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.307324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.307353 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.307355 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk2q2\" (UniqueName: \"kubernetes.io/projected/0f943243-e356-4d94-8635-709576aa00e6-kube-api-access-nk2q2\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.307510 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.307494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f943243-e356-4d94-8635-709576aa00e6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.307592 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.307548 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.307672 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:17.307657 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 15:52:17.307724 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:17.307715 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls podName:0f943243-e356-4d94-8635-709576aa00e6 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:17.807697878 +0000 UTC m=+83.070716528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-5d754" (UID: "0f943243-e356-4d94-8635-709576aa00e6") : secret "prometheus-operator-tls" not found Jul 08 15:52:17.308214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.308192 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f943243-e356-4d94-8635-709576aa00e6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.309830 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.309814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.317871 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.317846 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk2q2\" (UniqueName: \"kubernetes.io/projected/0f943243-e356-4d94-8635-709576aa00e6-kube-api-access-nk2q2\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.358360 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.358310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-96wh4\"" Jul 08 15:52:17.367037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.367018 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7w2mq" Jul 08 15:52:17.509418 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.509390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7w2mq"] Jul 08 15:52:17.512126 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:17.512084 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6142acdd_6d9e_4979_aa6e_85ad14509660.slice/crio-e0046da3c12f91c67e0bfd8f5634d1cda4364ec51c1a0290b9f37b7779c2f353 WatchSource:0}: Error finding container e0046da3c12f91c67e0bfd8f5634d1cda4364ec51c1a0290b9f37b7779c2f353: Status 404 returned error can't find the container with id e0046da3c12f91c67e0bfd8f5634d1cda4364ec51c1a0290b9f37b7779c2f353 Jul 08 15:52:17.653694 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.653614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7w2mq" event={"ID":"6142acdd-6d9e-4979-aa6e-85ad14509660","Type":"ContainerStarted","Data":"e0046da3c12f91c67e0bfd8f5634d1cda4364ec51c1a0290b9f37b7779c2f353"} Jul 08 15:52:17.810654 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.810626 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:17.813067 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:17.813042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f943243-e356-4d94-8635-709576aa00e6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5d754\" (UID: \"0f943243-e356-4d94-8635-709576aa00e6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:18.089515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:18.089447 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" Jul 08 15:52:18.259448 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:18.259416 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5d754"] Jul 08 15:52:18.661144 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:18.661089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" event={"ID":"0f943243-e356-4d94-8635-709576aa00e6","Type":"ContainerStarted","Data":"e99be4e2295801f05f5abe28126e800aa7657c77bdf6f1a0d2d03160ae0c5a9e"} Jul 08 15:52:22.640371 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.640347 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c79v7" Jul 08 15:52:22.680883 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.680856 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" event={"ID":"0f943243-e356-4d94-8635-709576aa00e6","Type":"ContainerStarted","Data":"1978851ecd24e8f8f398432c428d0a31daa3a35db55313ff5e05d1335947eefc"} Jul 08 15:52:22.681128 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.681094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" event={"ID":"0f943243-e356-4d94-8635-709576aa00e6","Type":"ContainerStarted","Data":"5979e8b8245f321d9fef87d7cf41bca49bc86c24c2c2cbacc31e12a0aa5a2b5a"} Jul 08 15:52:22.682840 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.682813 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7w2mq" event={"ID":"6142acdd-6d9e-4979-aa6e-85ad14509660","Type":"ContainerStarted","Data":"d92596d1990824fd443a186eac3db6be4ea6b765b944bcf1950688d2f1e87234"} Jul 08 15:52:22.682951 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.682868 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7w2mq" event={"ID":"6142acdd-6d9e-4979-aa6e-85ad14509660","Type":"ContainerStarted","Data":"fddca2f1438123483fc32ec36124963abb127d357ff3c6c6bd53b61416ac6880"} Jul 08 15:52:22.699595 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:22.698458 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5d754" podStartSLOduration=2.246088653 podStartE2EDuration="5.698440035s" totalImagesPullingTime="3.452351382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:22.696809386 +0000 UTC m=+87.959828089" watchObservedRunningTime="2026-07-08 15:52:22.698440035 +0000 UTC m=+87.961458685" Jul 08 15:52:24.558739 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.558706 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vx22t"] Jul 08 15:52:24.565056 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.565028 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.567351 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.567324 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 15:52:24.567490 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.567352 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 15:52:24.567679 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.567656 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4sr77\"" Jul 08 15:52:24.567880 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.567862 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 15:52:24.663752 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663721 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-metrics-client-ca\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.663919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663766 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.663919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjs46\" (UniqueName: \"kubernetes.io/projected/f32eb20b-3c70-4905-9780-0b3d25b6e806-kube-api-access-xjs46\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.663919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-sys\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.663919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663849 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.664114 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-root\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.664114 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.663987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.664114 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.664010 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-textfile\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.664114 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.664070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-wtmp\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765342 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-textfile\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-wtmp\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-metrics-client-ca\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjs46\" (UniqueName: \"kubernetes.io/projected/f32eb20b-3c70-4905-9780-0b3d25b6e806-kube-api-access-xjs46\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-sys\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765515 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.765834 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.765558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-root\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766088 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-wtmp\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766187 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766135 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-sys\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766187 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766166 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766507 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766481 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-textfile\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766843 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766816 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-metrics-client-ca\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766921 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766871 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f32eb20b-3c70-4905-9780-0b3d25b6e806-root\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.766988 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:24.766971 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 15:52:24.766988 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.766973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.767099 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:24.767029 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls podName:f32eb20b-3c70-4905-9780-0b3d25b6e806 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:25.267010215 +0000 UTC m=+90.530028863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls") pod "node-exporter-vx22t" (UID: "f32eb20b-3c70-4905-9780-0b3d25b6e806") : secret "node-exporter-tls" not found Jul 08 15:52:24.769965 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.769938 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:24.775454 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:24.775430 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjs46\" (UniqueName: \"kubernetes.io/projected/f32eb20b-3c70-4905-9780-0b3d25b6e806-kube-api-access-xjs46\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:25.270835 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:25.270803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:25.270959 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:25.270917 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 15:52:25.271013 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:25.270972 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls podName:f32eb20b-3c70-4905-9780-0b3d25b6e806 nodeName:}" failed. No retries permitted until 2026-07-08 15:52:26.270957497 +0000 UTC m=+91.533976148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls") pod "node-exporter-vx22t" (UID: "f32eb20b-3c70-4905-9780-0b3d25b6e806") : secret "node-exporter-tls" not found Jul 08 15:52:25.696919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:25.696886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7w2mq" event={"ID":"6142acdd-6d9e-4979-aa6e-85ad14509660","Type":"ContainerStarted","Data":"1514a8877d64a5ec2c70cc3bd02fb82fcaa5b0b8de97814acd1b38bc145029bf"} Jul 08 15:52:25.715727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:25.715673 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7w2mq" podStartSLOduration=33.272461891 podStartE2EDuration="40.715630469s" totalImagesPullingTime="7.443168578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 15:51:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:25.714682313 +0000 UTC m=+90.977700983" watchObservedRunningTime="2026-07-08 15:52:25.715630469 +0000 UTC m=+90.978649140" Jul 08 15:52:26.278884 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:26.278846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:26.281273 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:26.281249 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f32eb20b-3c70-4905-9780-0b3d25b6e806-node-exporter-tls\") pod \"node-exporter-vx22t\" (UID: \"f32eb20b-3c70-4905-9780-0b3d25b6e806\") " pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:26.377027 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:26.376996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vx22t" Jul 08 15:52:26.701599 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:26.701542 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx22t" event={"ID":"f32eb20b-3c70-4905-9780-0b3d25b6e806","Type":"ContainerStarted","Data":"792f0ef08bf899d765ed999b7ead4a66130ea46905f6d024a55de30216f922be"} Jul 08 15:52:27.707341 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:27.707252 2566 generic.go:356] "Generic (PLEG): container finished" podID="f32eb20b-3c70-4905-9780-0b3d25b6e806" containerID="7473df5cf22a881f0d955a27ea48a0aada7d3be343abaa568e0d944c76a373a3" exitCode=0 Jul 08 15:52:27.707341 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:27.707296 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx22t" event={"ID":"f32eb20b-3c70-4905-9780-0b3d25b6e806","Type":"ContainerDied","Data":"7473df5cf22a881f0d955a27ea48a0aada7d3be343abaa568e0d944c76a373a3"} Jul 08 15:52:28.713155 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.713119 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx22t" event={"ID":"f32eb20b-3c70-4905-9780-0b3d25b6e806","Type":"ContainerStarted","Data":"ab04b3da30a4c506cbf9912d9a9b5339976a01df1314ad80ee99647543615ad2"} Jul 08 15:52:28.713641 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.713170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx22t" event={"ID":"f32eb20b-3c70-4905-9780-0b3d25b6e806","Type":"ContainerStarted","Data":"537193050aa0b37692646bc5d6534becb717f7e7f9c0e161558ed59d63b5c063"} Jul 08 15:52:28.735726 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.735679 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vx22t" podStartSLOduration=3.621821237 podStartE2EDuration="4.735664311s" totalImagesPullingTime="1.042925115s" totalInitContainerRuntime="70.917959ms" isStatefulPod=true podCreationTimestamp="2026-07-08 15:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:28.734812764 +0000 UTC m=+93.997831434" watchObservedRunningTime="2026-07-08 15:52:28.735664311 +0000 UTC m=+93.998682980" Jul 08 15:52:28.840152 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.840117 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65cdc46999-hqkvm"] Jul 08 15:52:28.842218 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.842196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.844682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.844655 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 15:52:28.844782 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.844701 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-k4z6q\"" Jul 08 15:52:28.844844 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.844781 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 15:52:28.844916 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.844903 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 15:52:28.845073 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.845057 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4of0hdc2t61eh\"" Jul 08 15:52:28.845245 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.845233 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 15:52:28.853324 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.853302 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65cdc46999-hqkvm"] Jul 08 15:52:28.897781 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-client-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.897781 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-client-certs\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.897978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9e767ce-0def-4b36-967d-0f72f0314e79-audit-log\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.897978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk4kl\" (UniqueName: \"kubernetes.io/projected/f9e767ce-0def-4b36-967d-0f72f0314e79-kube-api-access-hk4kl\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.897978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-metrics-server-audit-profiles\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.897978 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-tls\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.898093 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.897994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999226 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-client-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999226 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-client-certs\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999226 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9e767ce-0def-4b36-967d-0f72f0314e79-audit-log\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999469 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk4kl\" (UniqueName: \"kubernetes.io/projected/f9e767ce-0def-4b36-967d-0f72f0314e79-kube-api-access-hk4kl\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999469 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-metrics-server-audit-profiles\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999469 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-tls\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999469 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:28.999725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:28.999702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9e767ce-0def-4b36-967d-0f72f0314e79-audit-log\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.000118 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.000091 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.000328 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.000310 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9e767ce-0def-4b36-967d-0f72f0314e79-metrics-server-audit-profiles\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.001961 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.001938 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-client-certs\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.002032 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.001977 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-secret-metrics-server-tls\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.002078 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.002048 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9e767ce-0def-4b36-967d-0f72f0314e79-client-ca-bundle\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.007136 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.007115 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk4kl\" (UniqueName: \"kubernetes.io/projected/f9e767ce-0def-4b36-967d-0f72f0314e79-kube-api-access-hk4kl\") pod \"metrics-server-65cdc46999-hqkvm\" (UID: \"f9e767ce-0def-4b36-967d-0f72f0314e79\") " pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.151786 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.151750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:29.295089 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.295062 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65cdc46999-hqkvm"] Jul 08 15:52:29.297514 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:29.297483 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e767ce_0def_4b36_967d_0f72f0314e79.slice/crio-d4d6ffa244f0fe16e70a0b81b33160ecc7957b1975c104fefd6be794d2786132 WatchSource:0}: Error finding container d4d6ffa244f0fe16e70a0b81b33160ecc7957b1975c104fefd6be794d2786132: Status 404 returned error can't find the container with id d4d6ffa244f0fe16e70a0b81b33160ecc7957b1975c104fefd6be794d2786132 Jul 08 15:52:29.361200 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.361163 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:52:29.364992 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.364969 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.367783 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.367759 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 15:52:29.367907 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.367857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfj5m\"" Jul 08 15:52:29.367953 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.367945 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 15:52:29.368045 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.368027 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 15:52:29.368139 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.368122 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 15:52:29.368449 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.368434 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 15:52:29.368535 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.368440 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 15:52:29.368739 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.368723 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 15:52:29.373426 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.373405 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 15:52:29.375425 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.375397 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:52:29.402245 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402225 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402350 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402350 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402350 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402536 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402536 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402437 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77zn\" (UniqueName: \"kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.402536 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.402480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503632 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503693 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77zn\" (UniqueName: \"kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503957 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.503957 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.503796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.504356 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.504334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.504653 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.504635 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.504721 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.504678 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.504887 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.504856 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.506096 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.506072 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.506207 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.506190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.512967 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.512947 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77zn\" (UniqueName: \"kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn\") pod \"console-7f7dfcbd5c-dj2ql\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.617746 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.617722 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:29.675622 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.675581 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:29.719182 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.719152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" event={"ID":"f9e767ce-0def-4b36-967d-0f72f0314e79","Type":"ContainerStarted","Data":"d4d6ffa244f0fe16e70a0b81b33160ecc7957b1975c104fefd6be794d2786132"} Jul 08 15:52:29.809261 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:29.809234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:52:29.812085 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:52:29.812050 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfac58c_8556_4e5d_89fd_03017e6a5b02.slice/crio-e2d3b1932a0b3730e2b5e5bf8aab3009ae8959d9289b2142cc1208f968bc2852 WatchSource:0}: Error finding container e2d3b1932a0b3730e2b5e5bf8aab3009ae8959d9289b2142cc1208f968bc2852: Status 404 returned error can't find the container with id e2d3b1932a0b3730e2b5e5bf8aab3009ae8959d9289b2142cc1208f968bc2852 Jul 08 15:52:30.725718 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:30.725642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7dfcbd5c-dj2ql" event={"ID":"9cfac58c-8556-4e5d-89fd-03017e6a5b02","Type":"ContainerStarted","Data":"e2d3b1932a0b3730e2b5e5bf8aab3009ae8959d9289b2142cc1208f968bc2852"} Jul 08 15:52:31.011253 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:31.011174 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:52:31.732362 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:31.732322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" event={"ID":"f9e767ce-0def-4b36-967d-0f72f0314e79","Type":"ContainerStarted","Data":"dbd2b0b0642d4593f431d13ea1af2fae32b4bad51eb9ebf861a25f4b1e5f2dcc"} Jul 08 15:52:31.753444 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:31.753397 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" podStartSLOduration=1.9148238 podStartE2EDuration="3.753377432s" totalImagesPullingTime="1.838553632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:31.752770506 +0000 UTC m=+97.015789176" watchObservedRunningTime="2026-07-08 15:52:31.753377432 +0000 UTC m=+97.016396101" Jul 08 15:52:32.571969 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:32.571939 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vlnjs" Jul 08 15:52:33.743090 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:33.743058 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7dfcbd5c-dj2ql" event={"ID":"9cfac58c-8556-4e5d-89fd-03017e6a5b02","Type":"ContainerStarted","Data":"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d"} Jul 08 15:52:33.765914 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:33.765858 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f7dfcbd5c-dj2ql" podStartSLOduration=1.7816478070000001 podStartE2EDuration="4.76583622s" totalImagesPullingTime="2.984188413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:52:33.764627746 +0000 UTC m=+99.027646415" watchObservedRunningTime="2026-07-08 15:52:33.76583622 +0000 UTC m=+99.028854892" Jul 08 15:52:34.592993 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:34.592964 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:52:39.675733 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:39.675700 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:52:49.152668 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:49.152618 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:49.153233 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:49.152734 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:52:54.823686 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:54.823653 2566 generic.go:356] "Generic (PLEG): container finished" podID="2ad66488-6def-4a0d-8f34-3a8a499f0800" containerID="33080194dfc42f2c1f812b14e84dc660004a824385c3bf38e30ab7e50dee3b90" exitCode=0 Jul 08 15:52:54.824045 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:54.823704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" event={"ID":"2ad66488-6def-4a0d-8f34-3a8a499f0800","Type":"ContainerDied","Data":"33080194dfc42f2c1f812b14e84dc660004a824385c3bf38e30ab7e50dee3b90"} Jul 08 15:52:54.824045 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:54.823960 2566 scope.go:117] "RemoveContainer" containerID="33080194dfc42f2c1f812b14e84dc660004a824385c3bf38e30ab7e50dee3b90" Jul 08 15:52:55.829256 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:55.829217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4q2t8" event={"ID":"2ad66488-6def-4a0d-8f34-3a8a499f0800","Type":"ContainerStarted","Data":"e10fec1d44e3ef4b7506d207f758ec466e7cb69c9f5e224372d13b821613424d"} Jul 08 15:52:56.040822 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.040759 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" podUID="244c193e-795a-4ed3-9c97-6a0007acec2e" containerName="registry" containerID="cri-o://d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70" gracePeriod=30 Jul 08 15:52:56.816803 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.816778 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:56.834385 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.834364 2566 generic.go:356] "Generic (PLEG): container finished" podID="244c193e-795a-4ed3-9c97-6a0007acec2e" containerID="d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70" exitCode=0 Jul 08 15:52:56.834744 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.834407 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" Jul 08 15:52:56.834744 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.834424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" event={"ID":"244c193e-795a-4ed3-9c97-6a0007acec2e","Type":"ContainerDied","Data":"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70"} Jul 08 15:52:56.834744 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.834460 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d648f84bb-dx2kk" event={"ID":"244c193e-795a-4ed3-9c97-6a0007acec2e","Type":"ContainerDied","Data":"5b1175a03cdf35299b024f3cddcb03dbf20721bf9e5756b685e0dbaa362d304a"} Jul 08 15:52:56.834744 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.834481 2566 scope.go:117] "RemoveContainer" containerID="d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70" Jul 08 15:52:56.846621 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.846483 2566 scope.go:117] "RemoveContainer" containerID="d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70" Jul 08 15:52:56.846992 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:52:56.846919 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70\": container with ID starting with d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70 not found: ID does not exist" containerID="d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70" Jul 08 15:52:56.846992 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.846954 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70"} err="failed to get container status \"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70\": rpc error: code = NotFound desc = could not find container \"d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70\": container with ID starting with d5bfb1c3af6eca08b48383e5aaaa377685daebdd7ccbe89347e951fde8e9fb70 not found: ID does not exist" Jul 08 15:52:56.931119 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931048 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931119 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931089 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4b5r\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931119 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931117 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931332 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931142 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931332 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931256 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931332 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931311 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931472 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931340 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931472 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931417 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted\") pod \"244c193e-795a-4ed3-9c97-6a0007acec2e\" (UID: \"244c193e-795a-4ed3-9c97-6a0007acec2e\") " Jul 08 15:52:56.931934 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931710 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:52:56.931934 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.931913 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:52:56.934178 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.934147 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:52:56.934178 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.934157 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:52:56.934306 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.934222 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r" (OuterVolumeSpecName: "kube-api-access-x4b5r") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "kube-api-access-x4b5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:52:56.934306 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.934282 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:52:56.934468 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.934448 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:52:56.939230 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:56.939209 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "244c193e-795a-4ed3-9c97-6a0007acec2e" (UID: "244c193e-795a-4ed3-9c97-6a0007acec2e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:52:57.032802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032781 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/244c193e-795a-4ed3-9c97-6a0007acec2e-ca-trust-extracted\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032802 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032799 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-installation-pull-secrets\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032809 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4b5r\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-kube-api-access-x4b5r\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032818 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/244c193e-795a-4ed3-9c97-6a0007acec2e-image-registry-private-configuration\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032826 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032834 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-trusted-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032841 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/244c193e-795a-4ed3-9c97-6a0007acec2e-registry-certificates\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.032919 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.032849 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/244c193e-795a-4ed3-9c97-6a0007acec2e-bound-sa-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:52:57.166612 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.166587 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:52:57.176700 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.176682 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d648f84bb-dx2kk"] Jul 08 15:52:57.268052 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:52:57.267995 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="244c193e-795a-4ed3-9c97-6a0007acec2e" path="/var/lib/kubelet/pods/244c193e-795a-4ed3-9c97-6a0007acec2e/volumes" Jul 08 15:53:00.771126 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:00.771064 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f7dfcbd5c-dj2ql" podUID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" containerName="console" containerID="cri-o://228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d" gracePeriod=15 Jul 08 15:53:01.017284 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.017261 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:53:01.160525 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160490 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160534 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160592 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160617 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d77zn\" (UniqueName: \"kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160654 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160683 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160713 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config\") pod \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\" (UID: \"9cfac58c-8556-4e5d-89fd-03017e6a5b02\") " Jul 08 15:53:01.160990 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.160952 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:53:01.161100 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.161044 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca" (OuterVolumeSpecName: "service-ca") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:53:01.161152 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.161127 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config" (OuterVolumeSpecName: "console-config") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:53:01.161186 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.161159 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:53:01.163083 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.163054 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:53:01.163083 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.163071 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:53:01.163216 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.163155 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn" (OuterVolumeSpecName: "kube-api-access-d77zn") pod "9cfac58c-8556-4e5d-89fd-03017e6a5b02" (UID: "9cfac58c-8556-4e5d-89fd-03017e6a5b02"). InnerVolumeSpecName "kube-api-access-d77zn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:53:01.261962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.261914 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-serving-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.261962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.261955 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-oauth-config\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.261962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.261969 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.262193 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.261983 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-trusted-ca-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.262193 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.261996 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-oauth-serving-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.262193 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.262008 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d77zn\" (UniqueName: \"kubernetes.io/projected/9cfac58c-8556-4e5d-89fd-03017e6a5b02-kube-api-access-d77zn\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.262193 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.262019 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cfac58c-8556-4e5d-89fd-03017e6a5b02-console-config\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:53:01.858403 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.858370 2566 generic.go:356] "Generic (PLEG): container finished" podID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" containerID="228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d" exitCode=0 Jul 08 15:53:01.858889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.858414 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7dfcbd5c-dj2ql" Jul 08 15:53:01.858889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.858461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7dfcbd5c-dj2ql" event={"ID":"9cfac58c-8556-4e5d-89fd-03017e6a5b02","Type":"ContainerDied","Data":"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d"} Jul 08 15:53:01.858889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.858559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7dfcbd5c-dj2ql" event={"ID":"9cfac58c-8556-4e5d-89fd-03017e6a5b02","Type":"ContainerDied","Data":"e2d3b1932a0b3730e2b5e5bf8aab3009ae8959d9289b2142cc1208f968bc2852"} Jul 08 15:53:01.858889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.858607 2566 scope.go:117] "RemoveContainer" containerID="228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d" Jul 08 15:53:01.866797 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.866779 2566 scope.go:117] "RemoveContainer" containerID="228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d" Jul 08 15:53:01.867089 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:53:01.867066 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d\": container with ID starting with 228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d not found: ID does not exist" containerID="228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d" Jul 08 15:53:01.867153 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.867098 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d"} err="failed to get container status \"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d\": rpc error: code = NotFound desc = could not find container \"228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d\": container with ID starting with 228f172410925f581ebf0dc98372b2337f81e7c02095c24dc75b7703c251062d not found: ID does not exist" Jul 08 15:53:01.876970 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.876949 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:53:01.881987 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:01.881967 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f7dfcbd5c-dj2ql"] Jul 08 15:53:03.267971 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:03.267940 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" path="/var/lib/kubelet/pods/9cfac58c-8556-4e5d-89fd-03017e6a5b02/volumes" Jul 08 15:53:09.156964 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:09.156934 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:53:09.160646 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:09.160613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65cdc46999-hqkvm" Jul 08 15:53:09.890414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:09.890385 2566 generic.go:356] "Generic (PLEG): container finished" podID="9ee637f4-2915-4fd8-9ce8-58d4629dbf65" containerID="321775c88dd8ae90830d0f4e90798fe33e630a3d61a1b1f9aff3a099216f49e7" exitCode=0 Jul 08 15:53:09.890641 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:09.890461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" event={"ID":"9ee637f4-2915-4fd8-9ce8-58d4629dbf65","Type":"ContainerDied","Data":"321775c88dd8ae90830d0f4e90798fe33e630a3d61a1b1f9aff3a099216f49e7"} Jul 08 15:53:09.890957 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:09.890939 2566 scope.go:117] "RemoveContainer" containerID="321775c88dd8ae90830d0f4e90798fe33e630a3d61a1b1f9aff3a099216f49e7" Jul 08 15:53:10.895199 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:10.895166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-klzjd" event={"ID":"9ee637f4-2915-4fd8-9ce8-58d4629dbf65","Type":"ContainerStarted","Data":"df1afb152eb31a9a708507bc2d20a568f4a37ebcb2d99f53423e81b04a3dac9d"} Jul 08 15:53:18.924655 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:18.924624 2566 generic.go:356] "Generic (PLEG): container finished" podID="d6d0f68f-8bae-4925-afb4-bc71338d6599" containerID="f4bb95f376743f9d4304179d955fd6fd1641a2f17471c7fa3dd07d47e81c8c6c" exitCode=0 Jul 08 15:53:18.924936 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:18.924703 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" event={"ID":"d6d0f68f-8bae-4925-afb4-bc71338d6599","Type":"ContainerDied","Data":"f4bb95f376743f9d4304179d955fd6fd1641a2f17471c7fa3dd07d47e81c8c6c"} Jul 08 15:53:18.925029 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:18.925016 2566 scope.go:117] "RemoveContainer" containerID="f4bb95f376743f9d4304179d955fd6fd1641a2f17471c7fa3dd07d47e81c8c6c" Jul 08 15:53:19.930667 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:53:19.930630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wbc58" event={"ID":"d6d0f68f-8bae-4925-afb4-bc71338d6599","Type":"ContainerStarted","Data":"2121a3270f391b3dfbc78457ca0954337c650d10a0443e1710f70c7376b810e8"} Jul 08 15:54:00.175811 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.175781 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:54:00.176202 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176132 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" containerName="console" Jul 08 15:54:00.176202 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176147 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" containerName="console" Jul 08 15:54:00.176202 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176160 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244c193e-795a-4ed3-9c97-6a0007acec2e" containerName="registry" Jul 08 15:54:00.176202 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176170 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="244c193e-795a-4ed3-9c97-6a0007acec2e" containerName="registry" Jul 08 15:54:00.176390 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176216 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="244c193e-795a-4ed3-9c97-6a0007acec2e" containerName="registry" Jul 08 15:54:00.176390 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.176225 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cfac58c-8556-4e5d-89fd-03017e6a5b02" containerName="console" Jul 08 15:54:00.180196 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.180177 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.184221 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.184193 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 15:54:00.184998 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.184971 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 15:54:00.185205 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185185 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 15:54:00.185448 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185425 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 15:54:00.185632 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185611 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfj5m\"" Jul 08 15:54:00.185707 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185659 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 15:54:00.185854 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185837 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 15:54:00.186052 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.185842 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 15:54:00.189871 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.189851 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 15:54:00.191371 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.191351 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:54:00.272862 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.272823 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.272862 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.272863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlvqn\" (UniqueName: \"kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.273109 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.272898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.273109 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.272913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.273109 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.272963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.273109 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.273055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.273109 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.273099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374256 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374419 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374419 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374419 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374614 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlvqn\" (UniqueName: \"kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374676 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.374676 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.374646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.375131 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.375105 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.375317 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.375222 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.375410 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.375221 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.375550 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.375532 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.377719 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.377695 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.377803 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.377788 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.384514 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.384489 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlvqn\" (UniqueName: \"kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn\") pod \"console-6748fdd9dd-7vz7h\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.492451 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.492389 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:00.620815 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:00.620794 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:54:00.623392 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:54:00.623369 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044e10e3_3e8f_47d4_9c4b_cc217ef9396c.slice/crio-e99aea8f83741c64eb59dbe8268d2d6bb096102115d4152ef4ba078440b3fbe9 WatchSource:0}: Error finding container e99aea8f83741c64eb59dbe8268d2d6bb096102115d4152ef4ba078440b3fbe9: Status 404 returned error can't find the container with id e99aea8f83741c64eb59dbe8268d2d6bb096102115d4152ef4ba078440b3fbe9 Jul 08 15:54:01.082226 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:01.082193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6748fdd9dd-7vz7h" event={"ID":"044e10e3-3e8f-47d4-9c4b-cc217ef9396c","Type":"ContainerStarted","Data":"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57"} Jul 08 15:54:01.082599 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:01.082240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6748fdd9dd-7vz7h" event={"ID":"044e10e3-3e8f-47d4-9c4b-cc217ef9396c","Type":"ContainerStarted","Data":"e99aea8f83741c64eb59dbe8268d2d6bb096102115d4152ef4ba078440b3fbe9"} Jul 08 15:54:01.102455 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:01.102402 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6748fdd9dd-7vz7h" podStartSLOduration=1.1023857989999999 podStartE2EDuration="1.102385799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:54:01.101011322 +0000 UTC m=+186.364029990" watchObservedRunningTime="2026-07-08 15:54:01.102385799 +0000 UTC m=+186.365404467" Jul 08 15:54:10.493086 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:10.493050 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:10.493623 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:10.493137 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:10.497857 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:10.497835 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:54:11.122939 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:54:11.122909 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:55:05.385909 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.385870 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz"] Jul 08 15:55:05.389283 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.389261 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.391869 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.391847 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 15:55:05.391973 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.391906 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 15:55:05.392664 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.392648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lvfqw\"" Jul 08 15:55:05.398661 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.398640 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz"] Jul 08 15:55:05.451009 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.450972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.451170 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.451048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plsm\" (UniqueName: \"kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.451170 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.451079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.552385 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.552339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.552546 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.552417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plsm\" (UniqueName: \"kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.552546 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.552440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.552767 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.552746 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.552829 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.552772 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.561348 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.561325 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plsm\" (UniqueName: \"kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.699200 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.699125 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:05.828346 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:05.828312 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz"] Jul 08 15:55:05.831557 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:55:05.831532 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8529218_b86e_4d98_b7b9_3306e2627ba5.slice/crio-fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107 WatchSource:0}: Error finding container fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107: Status 404 returned error can't find the container with id fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107 Jul 08 15:55:06.320247 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:06.320213 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" event={"ID":"c8529218-b86e-4d98-b7b9-3306e2627ba5","Type":"ContainerStarted","Data":"fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107"} Jul 08 15:55:14.353847 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:14.353812 2566 generic.go:356] "Generic (PLEG): container finished" podID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerID="03dfe08a5738898e008164ad5ce4bcb4fb9078293dba2d683fa77e6c864e1005" exitCode=0 Jul 08 15:55:14.354248 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:14.353908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" event={"ID":"c8529218-b86e-4d98-b7b9-3306e2627ba5","Type":"ContainerDied","Data":"03dfe08a5738898e008164ad5ce4bcb4fb9078293dba2d683fa77e6c864e1005"} Jul 08 15:55:17.366474 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:17.366433 2566 generic.go:356] "Generic (PLEG): container finished" podID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerID="66e0726c1d0294b609c5bf279566660515ea6bd5515e4b8600202b396325bf18" exitCode=0 Jul 08 15:55:17.366876 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:17.366483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" event={"ID":"c8529218-b86e-4d98-b7b9-3306e2627ba5","Type":"ContainerDied","Data":"66e0726c1d0294b609c5bf279566660515ea6bd5515e4b8600202b396325bf18"} Jul 08 15:55:26.405247 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:26.405210 2566 generic.go:356] "Generic (PLEG): container finished" podID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerID="22e6e2b6a1401d11f4d35f83f0fc5750dac71f6f3c6b4292ac94d3e53dd30097" exitCode=0 Jul 08 15:55:26.405644 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:26.405293 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" event={"ID":"c8529218-b86e-4d98-b7b9-3306e2627ba5","Type":"ContainerDied","Data":"22e6e2b6a1401d11f4d35f83f0fc5750dac71f6f3c6b4292ac94d3e53dd30097"} Jul 08 15:55:27.569975 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.569945 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:27.726502 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.726429 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle\") pod \"c8529218-b86e-4d98-b7b9-3306e2627ba5\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " Jul 08 15:55:27.726502 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.726481 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2plsm\" (UniqueName: \"kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm\") pod \"c8529218-b86e-4d98-b7b9-3306e2627ba5\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " Jul 08 15:55:27.726502 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.726499 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util\") pod \"c8529218-b86e-4d98-b7b9-3306e2627ba5\" (UID: \"c8529218-b86e-4d98-b7b9-3306e2627ba5\") " Jul 08 15:55:27.727090 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.727058 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle" (OuterVolumeSpecName: "bundle") pod "c8529218-b86e-4d98-b7b9-3306e2627ba5" (UID: "c8529218-b86e-4d98-b7b9-3306e2627ba5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:55:27.728805 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.728779 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm" (OuterVolumeSpecName: "kube-api-access-2plsm") pod "c8529218-b86e-4d98-b7b9-3306e2627ba5" (UID: "c8529218-b86e-4d98-b7b9-3306e2627ba5"). InnerVolumeSpecName "kube-api-access-2plsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:55:27.730795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.730775 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util" (OuterVolumeSpecName: "util") pod "c8529218-b86e-4d98-b7b9-3306e2627ba5" (UID: "c8529218-b86e-4d98-b7b9-3306e2627ba5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:55:27.827891 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.827864 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:27.827891 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.827892 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2plsm\" (UniqueName: \"kubernetes.io/projected/c8529218-b86e-4d98-b7b9-3306e2627ba5-kube-api-access-2plsm\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:27.828019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:27.827903 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8529218-b86e-4d98-b7b9-3306e2627ba5-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:28.414241 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:28.414206 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" event={"ID":"c8529218-b86e-4d98-b7b9-3306e2627ba5","Type":"ContainerDied","Data":"fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107"} Jul 08 15:55:28.414241 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:28.414224 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rnmdz" Jul 08 15:55:28.414499 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:28.414254 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd84a8937c8fcf009701e515c488ea35efcbff0ef0ebc441178e4ec271a87107" Jul 08 15:55:33.553402 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553373 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f"] Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553651 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="util" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553663 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="util" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553670 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="pull" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553675 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="pull" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553685 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="extract" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553691 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="extract" Jul 08 15:55:33.553861 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.553739 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8529218-b86e-4d98-b7b9-3306e2627ba5" containerName="extract" Jul 08 15:55:33.556951 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.556937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.559454 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.559431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 15:55:33.559783 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.559767 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vmk6v\"" Jul 08 15:55:33.560407 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.560390 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:55:33.571698 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.571676 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f"] Jul 08 15:55:33.670440 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.670412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlqq5\" (UniqueName: \"kubernetes.io/projected/7fcd5972-7b25-4ec8-be70-3a93ef738188-kube-api-access-zlqq5\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.670545 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.670486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fcd5972-7b25-4ec8-be70-3a93ef738188-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.771193 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.771166 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlqq5\" (UniqueName: \"kubernetes.io/projected/7fcd5972-7b25-4ec8-be70-3a93ef738188-kube-api-access-zlqq5\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.771282 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.771235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fcd5972-7b25-4ec8-be70-3a93ef738188-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.771559 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.771544 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fcd5972-7b25-4ec8-be70-3a93ef738188-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.779670 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.779646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlqq5\" (UniqueName: \"kubernetes.io/projected/7fcd5972-7b25-4ec8-be70-3a93ef738188-kube-api-access-zlqq5\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgd5f\" (UID: \"7fcd5972-7b25-4ec8-be70-3a93ef738188\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.865650 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.865631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" Jul 08 15:55:33.994533 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:33.994460 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f"] Jul 08 15:55:33.997538 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:55:33.997513 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcd5972_7b25_4ec8_be70_3a93ef738188.slice/crio-6e337c1ee1ee017fae68d68be312319608e32758774131e0ab0a94cd55b0076a WatchSource:0}: Error finding container 6e337c1ee1ee017fae68d68be312319608e32758774131e0ab0a94cd55b0076a: Status 404 returned error can't find the container with id 6e337c1ee1ee017fae68d68be312319608e32758774131e0ab0a94cd55b0076a Jul 08 15:55:34.437171 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:34.437134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" event={"ID":"7fcd5972-7b25-4ec8-be70-3a93ef738188","Type":"ContainerStarted","Data":"6e337c1ee1ee017fae68d68be312319608e32758774131e0ab0a94cd55b0076a"} Jul 08 15:55:36.447555 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:36.447507 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" event={"ID":"7fcd5972-7b25-4ec8-be70-3a93ef738188","Type":"ContainerStarted","Data":"d864b23c17a9295e09f30cff66895047534e1672f16a425353cabf23a7fa49b2"} Jul 08 15:55:36.467281 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:36.467220 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgd5f" podStartSLOduration=1.453701642 podStartE2EDuration="3.467199249s" totalImagesPullingTime="2.013497607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:55:36.465607466 +0000 UTC m=+281.728626139" watchObservedRunningTime="2026-07-08 15:55:36.467199249 +0000 UTC m=+281.730217918" Jul 08 15:55:37.320468 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.320393 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zxd2b"] Jul 08 15:55:37.322495 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.322479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.324680 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.324660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 15:55:37.324963 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.324948 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 15:55:37.328457 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.328437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8v56x\"" Jul 08 15:55:37.336991 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.336971 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zxd2b"] Jul 08 15:55:37.475847 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.475814 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk"] Jul 08 15:55:37.478172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.478156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.480852 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.480829 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 15:55:37.480979 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.480870 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 15:55:37.480979 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.480887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lvfqw\"" Jul 08 15:55:37.497255 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.497228 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk"] Jul 08 15:55:37.503026 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.503003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr5g8\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-kube-api-access-jr5g8\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.503143 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.503071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.604198 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.604173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.604321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.604212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.604321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.604258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr5g8\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-kube-api-access-jr5g8\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.604321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.604290 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.604321 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.604308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5w5\" (UniqueName: \"kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.618538 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.618505 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.618647 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.618610 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr5g8\" (UniqueName: \"kubernetes.io/projected/f85b3528-7136-4fc9-9af7-8cc881593b12-kube-api-access-jr5g8\") pod \"cert-manager-webhook-6486ffd997-zxd2b\" (UID: \"f85b3528-7136-4fc9-9af7-8cc881593b12\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.649981 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.649960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:37.704962 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.704936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.705070 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.704969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5w5\" (UniqueName: \"kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.705070 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.705017 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.705349 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.705327 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.705461 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.705406 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.716859 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.716834 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5w5\" (UniqueName: \"kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.783083 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.783060 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zxd2b"] Jul 08 15:55:37.785369 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:55:37.785343 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf85b3528_7136_4fc9_9af7_8cc881593b12.slice/crio-2a0a076b52d41f4cdbdc74d098826c96207bb349e978b74260c9622f59a19fe4 WatchSource:0}: Error finding container 2a0a076b52d41f4cdbdc74d098826c96207bb349e978b74260c9622f59a19fe4: Status 404 returned error can't find the container with id 2a0a076b52d41f4cdbdc74d098826c96207bb349e978b74260c9622f59a19fe4 Jul 08 15:55:37.786660 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.786639 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:37.927124 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:37.927094 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk"] Jul 08 15:55:37.929965 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:55:37.929940 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4b7759_22a8_4aeb_902c_23db4a3639ad.slice/crio-bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a WatchSource:0}: Error finding container bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a: Status 404 returned error can't find the container with id bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a Jul 08 15:55:38.456470 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:38.456436 2566 generic.go:356] "Generic (PLEG): container finished" podID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerID="a0019303b3f7e2236f8159a1d84c6fb329f9ac273d8666047d5bb348d71e7af3" exitCode=0 Jul 08 15:55:38.456670 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:38.456520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" event={"ID":"0c4b7759-22a8-4aeb-902c-23db4a3639ad","Type":"ContainerDied","Data":"a0019303b3f7e2236f8159a1d84c6fb329f9ac273d8666047d5bb348d71e7af3"} Jul 08 15:55:38.456670 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:38.456602 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" event={"ID":"0c4b7759-22a8-4aeb-902c-23db4a3639ad","Type":"ContainerStarted","Data":"bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a"} Jul 08 15:55:38.457763 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:38.457739 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" event={"ID":"f85b3528-7136-4fc9-9af7-8cc881593b12","Type":"ContainerStarted","Data":"2a0a076b52d41f4cdbdc74d098826c96207bb349e978b74260c9622f59a19fe4"} Jul 08 15:55:40.008319 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.008282 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn"] Jul 08 15:55:40.010988 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.010966 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.013262 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.013241 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-d85ns\"" Jul 08 15:55:40.022844 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.022800 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn"] Jul 08 15:55:40.125411 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.125375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmzl\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-kube-api-access-qwmzl\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.125554 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.125428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.226443 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.226406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmzl\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-kube-api-access-qwmzl\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.226839 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.226476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.235371 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.235340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.235524 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.235503 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmzl\" (UniqueName: \"kubernetes.io/projected/11dc3b1e-59d5-4c78-ae36-61c6046490c4-kube-api-access-qwmzl\") pod \"cert-manager-cainjector-559cdd8c87-5mmcn\" (UID: \"11dc3b1e-59d5-4c78-ae36-61c6046490c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.324520 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.324449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" Jul 08 15:55:40.606230 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:40.606208 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn"] Jul 08 15:55:40.608022 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:55:40.607998 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11dc3b1e_59d5_4c78_ae36_61c6046490c4.slice/crio-c09dacbdc4be3a4645a51c92f07e41e062ef4a2071e0330e0508ad3b72cae362 WatchSource:0}: Error finding container c09dacbdc4be3a4645a51c92f07e41e062ef4a2071e0330e0508ad3b72cae362: Status 404 returned error can't find the container with id c09dacbdc4be3a4645a51c92f07e41e062ef4a2071e0330e0508ad3b72cae362 Jul 08 15:55:41.474166 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.474131 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" event={"ID":"11dc3b1e-59d5-4c78-ae36-61c6046490c4","Type":"ContainerStarted","Data":"866487e4b0d99a1aa61f6b2cb4f1142931a40d377a06f0e7d354968bac55b7c4"} Jul 08 15:55:41.474629 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.474188 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" event={"ID":"11dc3b1e-59d5-4c78-ae36-61c6046490c4","Type":"ContainerStarted","Data":"c09dacbdc4be3a4645a51c92f07e41e062ef4a2071e0330e0508ad3b72cae362"} Jul 08 15:55:41.475992 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.475967 2566 generic.go:356] "Generic (PLEG): container finished" podID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerID="b815a1bb28d573ce7f3e625d409fe42545a1152b7bac02b7b5c7992745c5497a" exitCode=0 Jul 08 15:55:41.476101 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.476046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" event={"ID":"0c4b7759-22a8-4aeb-902c-23db4a3639ad","Type":"ContainerDied","Data":"b815a1bb28d573ce7f3e625d409fe42545a1152b7bac02b7b5c7992745c5497a"} Jul 08 15:55:41.477708 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.477661 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" event={"ID":"f85b3528-7136-4fc9-9af7-8cc881593b12","Type":"ContainerStarted","Data":"f6c9ed2231a4be10689dab3902056a8237757cffb2fb8da7975417ffd4aae7a5"} Jul 08 15:55:41.477838 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.477821 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:41.492997 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.492839 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5mmcn" podStartSLOduration=2.492822786 podStartE2EDuration="2.492822786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:55:41.491438814 +0000 UTC m=+286.754457505" watchObservedRunningTime="2026-07-08 15:55:41.492822786 +0000 UTC m=+286.755841454" Jul 08 15:55:41.527746 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:41.527699 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" podStartSLOduration=1.809367567 podStartE2EDuration="4.527679874s" totalImagesPullingTime="2.718312307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:55:41.527054484 +0000 UTC m=+286.790073165" watchObservedRunningTime="2026-07-08 15:55:41.527679874 +0000 UTC m=+286.790698545" Jul 08 15:55:42.483662 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:42.483628 2566 generic.go:356] "Generic (PLEG): container finished" podID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerID="b42172c1dd3bbb61f1eb5e8c816f8221bd0ed0d5820690fefd568eb9a3d8e846" exitCode=0 Jul 08 15:55:42.484000 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:42.483717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" event={"ID":"0c4b7759-22a8-4aeb-902c-23db4a3639ad","Type":"ContainerDied","Data":"b42172c1dd3bbb61f1eb5e8c816f8221bd0ed0d5820690fefd568eb9a3d8e846"} Jul 08 15:55:43.615399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.615373 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:43.754535 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.754463 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh5w5\" (UniqueName: \"kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5\") pod \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " Jul 08 15:55:43.754685 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.754560 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util\") pod \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " Jul 08 15:55:43.754685 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.754606 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle\") pod \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\" (UID: \"0c4b7759-22a8-4aeb-902c-23db4a3639ad\") " Jul 08 15:55:43.755037 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.755003 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle" (OuterVolumeSpecName: "bundle") pod "0c4b7759-22a8-4aeb-902c-23db4a3639ad" (UID: "0c4b7759-22a8-4aeb-902c-23db4a3639ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:55:43.756709 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.756681 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5" (OuterVolumeSpecName: "kube-api-access-dh5w5") pod "0c4b7759-22a8-4aeb-902c-23db4a3639ad" (UID: "0c4b7759-22a8-4aeb-902c-23db4a3639ad"). InnerVolumeSpecName "kube-api-access-dh5w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:55:43.759239 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.759212 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util" (OuterVolumeSpecName: "util") pod "0c4b7759-22a8-4aeb-902c-23db4a3639ad" (UID: "0c4b7759-22a8-4aeb-902c-23db4a3639ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:55:43.855652 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.855631 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh5w5\" (UniqueName: \"kubernetes.io/projected/0c4b7759-22a8-4aeb-902c-23db4a3639ad-kube-api-access-dh5w5\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:43.855652 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.855651 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:43.855812 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:43.855661 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4b7759-22a8-4aeb-902c-23db4a3639ad-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:55:44.494267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:44.494233 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" event={"ID":"0c4b7759-22a8-4aeb-902c-23db4a3639ad","Type":"ContainerDied","Data":"bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a"} Jul 08 15:55:44.494267 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:44.494267 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f626lk" Jul 08 15:55:44.494477 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:44.494284 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbc6018671e7684cfde1bceeec5e22b32bc3ca6297e49e5c2b9fc280a839163a" Jul 08 15:55:47.486305 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:47.486272 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zxd2b" Jul 08 15:55:55.127366 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:55.127339 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 15:55:55.127829 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:55.127724 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 15:55:55.133417 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:55.133390 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:55:55.133538 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:55.133513 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 15:55:55.138356 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:55:55.138342 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 15:56:02.971012 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.970974 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j"] Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971291 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="pull" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971301 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="pull" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971312 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="extract" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971317 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="extract" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971341 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="util" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971346 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="util" Jul 08 15:56:02.972795 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.971391 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4b7759-22a8-4aeb-902c-23db4a3639ad" containerName="extract" Jul 08 15:56:02.973611 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.973596 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:02.975969 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.975949 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 15:56:02.976715 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.976697 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 15:56:02.976770 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.976725 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lvfqw\"" Jul 08 15:56:02.987223 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.987199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prgx6\" (UniqueName: \"kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:02.987333 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.987244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:02.987333 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.987268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:02.989021 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:02.989002 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j"] Jul 08 15:56:03.088146 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.088110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prgx6\" (UniqueName: \"kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.088315 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.088164 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.088315 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.088188 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.088529 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.088510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.088606 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.088512 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.101172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.101146 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prgx6\" (UniqueName: \"kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.282453 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.282381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:03.419523 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.419499 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j"] Jul 08 15:56:03.421461 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:56:03.421428 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb848c8d6_c107_4b2e_bf27_c4ed2b8c5378.slice/crio-90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64 WatchSource:0}: Error finding container 90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64: Status 404 returned error can't find the container with id 90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64 Jul 08 15:56:03.575791 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.575727 2566 generic.go:356] "Generic (PLEG): container finished" podID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerID="018b9449f1c3f1f1fa31ad0abd7e48c17e528bbdef2b92380e05b682dcf5cef3" exitCode=0 Jul 08 15:56:03.575920 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.575822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" event={"ID":"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378","Type":"ContainerDied","Data":"018b9449f1c3f1f1fa31ad0abd7e48c17e528bbdef2b92380e05b682dcf5cef3"} Jul 08 15:56:03.575920 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.575878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" event={"ID":"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378","Type":"ContainerStarted","Data":"90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64"} Jul 08 15:56:03.576215 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:03.576201 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 15:56:05.586736 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:05.586705 2566 generic.go:356] "Generic (PLEG): container finished" podID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerID="c53d1313d12e6a2889867bbb3c073d2d3bf0ea3edbf750dbe6af676d421729b6" exitCode=0 Jul 08 15:56:05.587121 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:05.586744 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" event={"ID":"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378","Type":"ContainerDied","Data":"c53d1313d12e6a2889867bbb3c073d2d3bf0ea3edbf750dbe6af676d421729b6"} Jul 08 15:56:06.592638 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:06.592604 2566 generic.go:356] "Generic (PLEG): container finished" podID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerID="ec5954b5cf41ebefe525de5b7c91f7636b8fb7c4fee45faeb8bc379193802a14" exitCode=0 Jul 08 15:56:06.593046 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:06.592686 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" event={"ID":"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378","Type":"ContainerDied","Data":"ec5954b5cf41ebefe525de5b7c91f7636b8fb7c4fee45faeb8bc379193802a14"} Jul 08 15:56:07.723016 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.722989 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:07.820210 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.820185 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle\") pod \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " Jul 08 15:56:07.820353 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.820227 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prgx6\" (UniqueName: \"kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6\") pod \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " Jul 08 15:56:07.820353 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.820250 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util\") pod \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\" (UID: \"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378\") " Jul 08 15:56:07.820911 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.820886 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle" (OuterVolumeSpecName: "bundle") pod "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" (UID: "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:56:07.822684 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.822660 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6" (OuterVolumeSpecName: "kube-api-access-prgx6") pod "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" (UID: "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378"). InnerVolumeSpecName "kube-api-access-prgx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:56:07.826131 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.826108 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util" (OuterVolumeSpecName: "util") pod "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" (UID: "b848c8d6-c107-4b2e-bf27-c4ed2b8c5378"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:56:07.921019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.920934 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prgx6\" (UniqueName: \"kubernetes.io/projected/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-kube-api-access-prgx6\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:56:07.921019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.920971 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:56:07.921019 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:07.920981 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b848c8d6-c107-4b2e-bf27-c4ed2b8c5378-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:56:08.608915 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:08.608885 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" Jul 08 15:56:08.608915 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:08.608899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbt9j" event={"ID":"b848c8d6-c107-4b2e-bf27-c4ed2b8c5378","Type":"ContainerDied","Data":"90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64"} Jul 08 15:56:08.609106 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:08.608940 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90d7530dedc28027e61f52905aa6f5156aca735e15e9a72a6e65cd8d8f199e64" Jul 08 15:56:14.413676 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413644 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2"] Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413944 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="util" Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413955 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="util" Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413968 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="pull" Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413973 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="pull" Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413985 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="extract" Jul 08 15:56:14.414022 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.413994 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="extract" Jul 08 15:56:14.414230 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.414042 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b848c8d6-c107-4b2e-bf27-c4ed2b8c5378" containerName="extract" Jul 08 15:56:14.418133 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.418114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.420706 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.420686 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 15:56:14.421410 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.421383 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 15:56:14.421410 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.421383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jcrsn\"" Jul 08 15:56:14.421609 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.421425 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 15:56:14.421609 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.421466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 15:56:14.421609 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.421471 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 15:56:14.430329 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.430309 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2"] Jul 08 15:56:14.475422 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.475397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.475512 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.475458 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4610c5db-c917-4822-bab3-36ae1c3b7d20-manager-config\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.475606 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.475518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rhxj\" (UniqueName: \"kubernetes.io/projected/4610c5db-c917-4822-bab3-36ae1c3b7d20-kube-api-access-9rhxj\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.475606 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.475551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-metrics-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.576091 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.576061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-metrics-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.576205 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.576102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.576256 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.576231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4610c5db-c917-4822-bab3-36ae1c3b7d20-manager-config\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.576314 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.576297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rhxj\" (UniqueName: \"kubernetes.io/projected/4610c5db-c917-4822-bab3-36ae1c3b7d20-kube-api-access-9rhxj\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.576875 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.576849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4610c5db-c917-4822-bab3-36ae1c3b7d20-manager-config\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.578680 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.578651 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.578812 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.578794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4610c5db-c917-4822-bab3-36ae1c3b7d20-metrics-cert\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.588752 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.588733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rhxj\" (UniqueName: \"kubernetes.io/projected/4610c5db-c917-4822-bab3-36ae1c3b7d20-kube-api-access-9rhxj\") pod \"lws-controller-manager-57b64f8b44-884b2\" (UID: \"4610c5db-c917-4822-bab3-36ae1c3b7d20\") " pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.727330 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.727252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:14.872080 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:14.872054 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2"] Jul 08 15:56:14.874091 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:56:14.874068 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4610c5db_c917_4822_bab3_36ae1c3b7d20.slice/crio-bea88d1928d482d6b29e549eae19016aef6064a36c8c987021caf8cfd0d382c8 WatchSource:0}: Error finding container bea88d1928d482d6b29e549eae19016aef6064a36c8c987021caf8cfd0d382c8: Status 404 returned error can't find the container with id bea88d1928d482d6b29e549eae19016aef6064a36c8c987021caf8cfd0d382c8 Jul 08 15:56:15.640884 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:15.640851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" event={"ID":"4610c5db-c917-4822-bab3-36ae1c3b7d20","Type":"ContainerStarted","Data":"bea88d1928d482d6b29e549eae19016aef6064a36c8c987021caf8cfd0d382c8"} Jul 08 15:56:17.652530 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:17.652496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" event={"ID":"4610c5db-c917-4822-bab3-36ae1c3b7d20","Type":"ContainerStarted","Data":"e29fd50938d98574d07273ac9ebb77ac69fb395886a3945057d2d430db04707c"} Jul 08 15:56:17.652907 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:17.652617 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:17.678172 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:17.678124 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" podStartSLOduration=1.648424591 podStartE2EDuration="3.678109977s" totalImagesPullingTime="2.029685386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:56:17.676998952 +0000 UTC m=+322.940017620" watchObservedRunningTime="2026-07-08 15:56:17.678109977 +0000 UTC m=+322.941128646" Jul 08 15:56:20.043386 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.043355 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98"] Jul 08 15:56:20.046852 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.046836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.049860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.049837 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 15:56:20.049860 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.049858 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 15:56:20.050068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.049914 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 15:56:20.050068 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.050033 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g2rr9\"" Jul 08 15:56:20.050203 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.050187 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 15:56:20.061360 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.061338 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98"] Jul 08 15:56:20.119447 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.119417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-manifests\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.119604 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.119449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.119604 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.119475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-264kj\" (UniqueName: \"kubernetes.io/projected/cc22f47f-0e1b-4340-9ee2-436085a849aa-kube-api-access-264kj\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.119693 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.119650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.119750 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.119697 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-tmp\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.220950 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.220904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.220950 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.220959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-tmp\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.221185 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.220996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-manifests\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.221185 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.221012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.221185 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.221030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-264kj\" (UniqueName: \"kubernetes.io/projected/cc22f47f-0e1b-4340-9ee2-436085a849aa-kube-api-access-264kj\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.221458 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.221439 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-manifests\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.223410 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.223387 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc22f47f-0e1b-4340-9ee2-436085a849aa-tmp\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.223507 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.223492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.223542 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.223496 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc22f47f-0e1b-4340-9ee2-436085a849aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.229674 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.229647 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-264kj\" (UniqueName: \"kubernetes.io/projected/cc22f47f-0e1b-4340-9ee2-436085a849aa-kube-api-access-264kj\") pod \"opendatahub-operator-controller-manager-86c94cb7c4-j6k98\" (UID: \"cc22f47f-0e1b-4340-9ee2-436085a849aa\") " pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.357945 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.357921 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:20.494697 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.494673 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98"] Jul 08 15:56:20.496970 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:56:20.496937 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc22f47f_0e1b_4340_9ee2_436085a849aa.slice/crio-e15b829e8e04219859dbd9ea4b0c845b1836fee7b111574ac92f6acd8982aaa8 WatchSource:0}: Error finding container e15b829e8e04219859dbd9ea4b0c845b1836fee7b111574ac92f6acd8982aaa8: Status 404 returned error can't find the container with id e15b829e8e04219859dbd9ea4b0c845b1836fee7b111574ac92f6acd8982aaa8 Jul 08 15:56:20.667746 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:20.667679 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" event={"ID":"cc22f47f-0e1b-4340-9ee2-436085a849aa","Type":"ContainerStarted","Data":"e15b829e8e04219859dbd9ea4b0c845b1836fee7b111574ac92f6acd8982aaa8"} Jul 08 15:56:23.684726 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:23.684690 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" event={"ID":"cc22f47f-0e1b-4340-9ee2-436085a849aa","Type":"ContainerStarted","Data":"12baed42949ef089f7bd857fd1dcd0f1b29dc480bc42c3a3b0c8ff430f2a0da4"} Jul 08 15:56:25.695792 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:25.695758 2566 generic.go:356] "Generic (PLEG): container finished" podID="cc22f47f-0e1b-4340-9ee2-436085a849aa" containerID="12baed42949ef089f7bd857fd1dcd0f1b29dc480bc42c3a3b0c8ff430f2a0da4" exitCode=0 Jul 08 15:56:25.696169 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:25.695831 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" event={"ID":"cc22f47f-0e1b-4340-9ee2-436085a849aa","Type":"ContainerDied","Data":"12baed42949ef089f7bd857fd1dcd0f1b29dc480bc42c3a3b0c8ff430f2a0da4"} Jul 08 15:56:26.707174 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:26.707141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" event={"ID":"cc22f47f-0e1b-4340-9ee2-436085a849aa","Type":"ContainerStarted","Data":"87ec3a10428b82623b9d30e289489a8b16f226e6d4ee977e03df1a0b0bd9db45"} Jul 08 15:56:26.707532 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:26.707376 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:26.730873 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:26.730824 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" podStartSLOduration=2.041162524 podStartE2EDuration="6.730807084s" totalImagesPullingTime="3.606636469s" totalInitContainerRuntime="1.083008091s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:56:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:56:26.728711151 +0000 UTC m=+331.991729831" watchObservedRunningTime="2026-07-08 15:56:26.730807084 +0000 UTC m=+331.993825752" Jul 08 15:56:28.659124 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:28.659095 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57b64f8b44-884b2" Jul 08 15:56:37.715036 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:37.715006 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86c94cb7c4-j6k98" Jul 08 15:56:48.215754 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.215677 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5"] Jul 08 15:56:48.218929 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.218912 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.222393 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.222372 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 15:56:48.222393 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.222389 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 15:56:48.222771 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.222755 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rrhkt\"" Jul 08 15:56:48.222850 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.222800 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 15:56:48.222900 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.222875 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 15:56:48.237389 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.237368 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5"] Jul 08 15:56:48.332066 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2rg\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-kube-api-access-np2rg\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332194 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332075 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332235 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332216 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332282 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332263 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332323 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332360 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332325 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.332360 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.332349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433153 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433126 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433304 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433304 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433376 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433376 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433453 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2rg\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-kube-api-access-np2rg\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.433549 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.433509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.434292 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.434253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.435727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.435700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.435820 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.435729 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.435983 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.435966 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.436043 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.436017 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.448119 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.448096 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.449179 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.449162 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2rg\" (UniqueName: \"kubernetes.io/projected/dcf49aaa-ecaa-4172-9f78-dbf73ea7e612-kube-api-access-np2rg\") pod \"istiod-openshift-gateway-94bb8fbfd-5t6g5\" (UID: \"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.527775 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.527721 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:48.683126 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.683098 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5"] Jul 08 15:56:48.684270 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:56:48.684242 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf49aaa_ecaa_4172_9f78_dbf73ea7e612.slice/crio-42846e9f265a0358e738b83491eb73f6a08952877d9e59f4cb2bf62a3acf00a4 WatchSource:0}: Error finding container 42846e9f265a0358e738b83491eb73f6a08952877d9e59f4cb2bf62a3acf00a4: Status 404 returned error can't find the container with id 42846e9f265a0358e738b83491eb73f6a08952877d9e59f4cb2bf62a3acf00a4 Jul 08 15:56:48.805625 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:48.805539 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" event={"ID":"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612","Type":"ContainerStarted","Data":"42846e9f265a0358e738b83491eb73f6a08952877d9e59f4cb2bf62a3acf00a4"} Jul 08 15:56:51.466891 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:51.466853 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 15:56:51.467144 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:51.466940 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 15:56:51.825369 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:51.825276 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" event={"ID":"dcf49aaa-ecaa-4172-9f78-dbf73ea7e612","Type":"ContainerStarted","Data":"ef13ac77b7e5dbbcc859235580326be0f32db4ab155a82d140a868b9e6e2730f"} Jul 08 15:56:51.825720 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:51.825370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:56:51.857691 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:51.857650 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" podStartSLOduration=1.077438127 podStartE2EDuration="3.857633612s" totalImagesPullingTime="2.780195485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:56:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:56:51.855376703 +0000 UTC m=+357.118395373" watchObservedRunningTime="2026-07-08 15:56:51.857633612 +0000 UTC m=+357.120652281" Jul 08 15:56:52.831179 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:56:52.831141 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5t6g5" Jul 08 15:57:56.352390 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.352357 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk"] Jul 08 15:57:56.355751 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.355736 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.358921 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.358902 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 15:57:56.359025 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.358908 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9xp52\"" Jul 08 15:57:56.359025 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.358938 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 15:57:56.366523 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.366502 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk"] Jul 08 15:57:56.463942 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.463918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.464091 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.463946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwwl8\" (UniqueName: \"kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.464091 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.464061 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.564425 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.564391 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.564596 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.564449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.564596 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.564473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwwl8\" (UniqueName: \"kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.564756 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.564738 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.564816 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.564793 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.573828 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.573809 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwwl8\" (UniqueName: \"kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.665110 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.665039 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:57:56.798220 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:56.798198 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk"] Jul 08 15:57:56.800391 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:57:56.800361 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f0b268_389a_437c_9ea6_29e7750b9553.slice/crio-fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b WatchSource:0}: Error finding container fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b: Status 404 returned error can't find the container with id fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b Jul 08 15:57:57.117655 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.117615 2566 generic.go:356] "Generic (PLEG): container finished" podID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerID="9860abd3d325fdc5effd4cd4ee0bb4354f4df6a1d1af64ad4aeab298a812e253" exitCode=0 Jul 08 15:57:57.117822 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.117709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" event={"ID":"a9f0b268-389a-437c-9ea6-29e7750b9553","Type":"ContainerDied","Data":"9860abd3d325fdc5effd4cd4ee0bb4354f4df6a1d1af64ad4aeab298a812e253"} Jul 08 15:57:57.117822 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.117752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" event={"ID":"a9f0b268-389a-437c-9ea6-29e7750b9553","Type":"ContainerStarted","Data":"fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b"} Jul 08 15:57:57.151748 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.151722 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt"] Jul 08 15:57:57.155784 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.155763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.164183 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.164160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt"] Jul 08 15:57:57.270791 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.270756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.270914 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.270801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.270914 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.270904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69kqq\" (UniqueName: \"kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.371851 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.371775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.371851 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.371818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.372265 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.372043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69kqq\" (UniqueName: \"kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.372265 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.372178 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.372265 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.372204 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.380791 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.380759 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69kqq\" (UniqueName: \"kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.465294 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.465270 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:57:57.545389 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.545356 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d"] Jul 08 15:57:57.549052 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.549030 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.558414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.558388 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d"] Jul 08 15:57:57.598649 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.598620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt"] Jul 08 15:57:57.601371 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:57:57.601338 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027d33d0_e0d8_475b_948a_c31fc3bb95b7.slice/crio-c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa WatchSource:0}: Error finding container c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa: Status 404 returned error can't find the container with id c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa Jul 08 15:57:57.674896 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.674871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.674987 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.674909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqx5z\" (UniqueName: \"kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.674987 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.674938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.775777 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.775751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.775889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.775797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqx5z\" (UniqueName: \"kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.775889 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.775827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.776134 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.776113 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.776134 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.776131 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.784090 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.784062 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqx5z\" (UniqueName: \"kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.862064 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.862037 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:57:57.953496 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.953474 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8"] Jul 08 15:57:57.958029 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.958009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:57.968558 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:57.968533 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8"] Jul 08 15:57:58.028892 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.028868 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d"] Jul 08 15:57:58.030758 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:57:58.030727 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2e37d4_2f38_466e_997b_b0ea9382fbb1.slice/crio-1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5 WatchSource:0}: Error finding container 1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5: Status 404 returned error can't find the container with id 1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5 Jul 08 15:57:58.077372 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.077343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kft\" (UniqueName: \"kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.077494 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.077387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.077494 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.077404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.124343 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.124318 2566 generic.go:356] "Generic (PLEG): container finished" podID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerID="aadb2897efe8367c94b4228f4488084a200c8a7f0bd152bcb7ad0f94dece9a75" exitCode=0 Jul 08 15:57:58.124439 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.124391 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" event={"ID":"a9f0b268-389a-437c-9ea6-29e7750b9553","Type":"ContainerDied","Data":"aadb2897efe8367c94b4228f4488084a200c8a7f0bd152bcb7ad0f94dece9a75"} Jul 08 15:57:58.126062 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.126038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" event={"ID":"9a2e37d4-2f38-466e-997b-b0ea9382fbb1","Type":"ContainerStarted","Data":"1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5"} Jul 08 15:57:58.127500 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.127477 2566 generic.go:356] "Generic (PLEG): container finished" podID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerID="9fe46578d62cb479c3bd59c0b0c2735999e5e9629d7e2bdb504f7c493455aaf4" exitCode=0 Jul 08 15:57:58.127595 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.127511 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerDied","Data":"9fe46578d62cb479c3bd59c0b0c2735999e5e9629d7e2bdb504f7c493455aaf4"} Jul 08 15:57:58.127595 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.127543 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerStarted","Data":"c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa"} Jul 08 15:57:58.178533 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.178506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kft\" (UniqueName: \"kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.178727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.178703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.178821 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.178742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.179103 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.179078 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.179166 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.179106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.187899 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.187875 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kft\" (UniqueName: \"kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.299080 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.299053 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:57:58.428936 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:58.428911 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8"] Jul 08 15:57:58.430252 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:57:58.430226 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7b3505_7b06_4171_8e1d_5c89caea98b3.slice/crio-795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b WatchSource:0}: Error finding container 795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b: Status 404 returned error can't find the container with id 795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b Jul 08 15:57:59.134834 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.134805 2566 generic.go:356] "Generic (PLEG): container finished" podID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerID="a4bc4bfda6608df002671014f4d12ea9424d862b96793d4eddf0e9a363bd86cc" exitCode=0 Jul 08 15:57:59.134949 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.134897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" event={"ID":"a9f0b268-389a-437c-9ea6-29e7750b9553","Type":"ContainerDied","Data":"a4bc4bfda6608df002671014f4d12ea9424d862b96793d4eddf0e9a363bd86cc"} Jul 08 15:57:59.136495 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.136471 2566 generic.go:356] "Generic (PLEG): container finished" podID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerID="2ac81b69650ce45a1177648ec95f94be2bfbd535594e722abec5040f7b4bc715" exitCode=0 Jul 08 15:57:59.136619 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.136560 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" event={"ID":"9a2e37d4-2f38-466e-997b-b0ea9382fbb1","Type":"ContainerDied","Data":"2ac81b69650ce45a1177648ec95f94be2bfbd535594e722abec5040f7b4bc715"} Jul 08 15:57:59.138016 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.137995 2566 generic.go:356] "Generic (PLEG): container finished" podID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerID="2b6e9792d04e7121b0e9a50fefe21563f02102fc321f8220efca8f5906ed251b" exitCode=0 Jul 08 15:57:59.138128 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.138110 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerDied","Data":"2b6e9792d04e7121b0e9a50fefe21563f02102fc321f8220efca8f5906ed251b"} Jul 08 15:57:59.138206 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.138150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerStarted","Data":"795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b"} Jul 08 15:57:59.140003 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:57:59.139982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerStarted","Data":"1d297bf4ecce810c036fa54da91c58d10f5474fdd45098a52e57991f97a2f4a5"} Jul 08 15:58:00.146881 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.146849 2566 generic.go:356] "Generic (PLEG): container finished" podID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerID="1d297bf4ecce810c036fa54da91c58d10f5474fdd45098a52e57991f97a2f4a5" exitCode=0 Jul 08 15:58:00.146881 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.146873 2566 generic.go:356] "Generic (PLEG): container finished" podID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerID="5f230e2c29730d32f530411fc00ef96c215ab461b49294b35cd559203af6f538" exitCode=0 Jul 08 15:58:00.147275 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.146930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerDied","Data":"1d297bf4ecce810c036fa54da91c58d10f5474fdd45098a52e57991f97a2f4a5"} Jul 08 15:58:00.147275 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.146981 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerDied","Data":"5f230e2c29730d32f530411fc00ef96c215ab461b49294b35cd559203af6f538"} Jul 08 15:58:00.148702 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.148678 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerStarted","Data":"e971ec4f931d81eeaefe11e50373be0a0aa73e7cc7d3ad096c7ef0b5bd6cef36"} Jul 08 15:58:00.281886 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.281865 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:58:00.402791 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.402767 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle\") pod \"a9f0b268-389a-437c-9ea6-29e7750b9553\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " Jul 08 15:58:00.402912 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.402866 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util\") pod \"a9f0b268-389a-437c-9ea6-29e7750b9553\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " Jul 08 15:58:00.402912 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.402888 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwwl8\" (UniqueName: \"kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8\") pod \"a9f0b268-389a-437c-9ea6-29e7750b9553\" (UID: \"a9f0b268-389a-437c-9ea6-29e7750b9553\") " Jul 08 15:58:00.403510 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.403481 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle" (OuterVolumeSpecName: "bundle") pod "a9f0b268-389a-437c-9ea6-29e7750b9553" (UID: "a9f0b268-389a-437c-9ea6-29e7750b9553"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:00.405092 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.405072 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8" (OuterVolumeSpecName: "kube-api-access-jwwl8") pod "a9f0b268-389a-437c-9ea6-29e7750b9553" (UID: "a9f0b268-389a-437c-9ea6-29e7750b9553"). InnerVolumeSpecName "kube-api-access-jwwl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:58:00.408557 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.408526 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util" (OuterVolumeSpecName: "util") pod "a9f0b268-389a-437c-9ea6-29e7750b9553" (UID: "a9f0b268-389a-437c-9ea6-29e7750b9553"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:00.503651 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.503623 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:00.503651 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.503645 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9f0b268-389a-437c-9ea6-29e7750b9553-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:00.503651 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:00.503654 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwwl8\" (UniqueName: \"kubernetes.io/projected/a9f0b268-389a-437c-9ea6-29e7750b9553-kube-api-access-jwwl8\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:01.155292 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.155260 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" event={"ID":"a9f0b268-389a-437c-9ea6-29e7750b9553","Type":"ContainerDied","Data":"fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b"} Jul 08 15:58:01.155727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.155304 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcaff9b4024de133992584a5c721db4f3f46e89299e6f88648705707fb027c0b" Jul 08 15:58:01.155727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.155272 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk" Jul 08 15:58:01.157126 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.157098 2566 generic.go:356] "Generic (PLEG): container finished" podID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerID="3100e8608892891e2d3108831328f1663d381abfd19ff2639d3b720a2118e2db" exitCode=0 Jul 08 15:58:01.157248 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.157178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" event={"ID":"9a2e37d4-2f38-466e-997b-b0ea9382fbb1","Type":"ContainerDied","Data":"3100e8608892891e2d3108831328f1663d381abfd19ff2639d3b720a2118e2db"} Jul 08 15:58:01.159180 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.159157 2566 generic.go:356] "Generic (PLEG): container finished" podID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerID="e971ec4f931d81eeaefe11e50373be0a0aa73e7cc7d3ad096c7ef0b5bd6cef36" exitCode=0 Jul 08 15:58:01.159298 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.159237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerDied","Data":"e971ec4f931d81eeaefe11e50373be0a0aa73e7cc7d3ad096c7ef0b5bd6cef36"} Jul 08 15:58:01.303555 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.303528 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:58:01.409621 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.409588 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util\") pod \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " Jul 08 15:58:01.409776 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.409641 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle\") pod \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " Jul 08 15:58:01.409776 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.409700 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69kqq\" (UniqueName: \"kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq\") pod \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\" (UID: \"027d33d0-e0d8-475b-948a-c31fc3bb95b7\") " Jul 08 15:58:01.410155 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.410132 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle" (OuterVolumeSpecName: "bundle") pod "027d33d0-e0d8-475b-948a-c31fc3bb95b7" (UID: "027d33d0-e0d8-475b-948a-c31fc3bb95b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:01.411875 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.411852 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq" (OuterVolumeSpecName: "kube-api-access-69kqq") pod "027d33d0-e0d8-475b-948a-c31fc3bb95b7" (UID: "027d33d0-e0d8-475b-948a-c31fc3bb95b7"). InnerVolumeSpecName "kube-api-access-69kqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:58:01.414367 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.414332 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util" (OuterVolumeSpecName: "util") pod "027d33d0-e0d8-475b-948a-c31fc3bb95b7" (UID: "027d33d0-e0d8-475b-948a-c31fc3bb95b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:01.510641 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.510592 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:01.510641 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.510637 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027d33d0-e0d8-475b-948a-c31fc3bb95b7-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:01.510641 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:01.510649 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69kqq\" (UniqueName: \"kubernetes.io/projected/027d33d0-e0d8-475b-948a-c31fc3bb95b7-kube-api-access-69kqq\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:02.166495 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.166455 2566 generic.go:356] "Generic (PLEG): container finished" podID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerID="3df50c9afb239d575a27c0bd76377a711a440a142b8508d3ac0983b15c9fa81e" exitCode=0 Jul 08 15:58:02.166973 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.166548 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" event={"ID":"9a2e37d4-2f38-466e-997b-b0ea9382fbb1","Type":"ContainerDied","Data":"3df50c9afb239d575a27c0bd76377a711a440a142b8508d3ac0983b15c9fa81e"} Jul 08 15:58:02.168373 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.168349 2566 generic.go:356] "Generic (PLEG): container finished" podID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerID="81c0f0d16062b378e062e74d33d9f70cb17409123a160f28460a68c8b453bb04" exitCode=0 Jul 08 15:58:02.168495 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.168434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerDied","Data":"81c0f0d16062b378e062e74d33d9f70cb17409123a160f28460a68c8b453bb04"} Jul 08 15:58:02.170281 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.170257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" event={"ID":"027d33d0-e0d8-475b-948a-c31fc3bb95b7","Type":"ContainerDied","Data":"c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa"} Jul 08 15:58:02.170387 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.170288 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt" Jul 08 15:58:02.170387 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:02.170290 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0735b446a93bc0e0bf72741859fed43ebd9d5c3e6a802304b3faf2695a586aa" Jul 08 15:58:03.333112 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.333086 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:58:03.335243 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.335226 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:58:03.425884 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.425813 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2kft\" (UniqueName: \"kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft\") pod \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " Jul 08 15:58:03.425884 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.425873 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle\") pod \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " Jul 08 15:58:03.426122 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.425911 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util\") pod \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\" (UID: \"cf7b3505-7b06-4171-8e1d-5c89caea98b3\") " Jul 08 15:58:03.426122 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.425939 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqx5z\" (UniqueName: \"kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z\") pod \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " Jul 08 15:58:03.426122 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.425988 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle\") pod \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " Jul 08 15:58:03.426122 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.426002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util\") pod \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\" (UID: \"9a2e37d4-2f38-466e-997b-b0ea9382fbb1\") " Jul 08 15:58:03.426518 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.426488 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle" (OuterVolumeSpecName: "bundle") pod "cf7b3505-7b06-4171-8e1d-5c89caea98b3" (UID: "cf7b3505-7b06-4171-8e1d-5c89caea98b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:03.426673 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.426536 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle" (OuterVolumeSpecName: "bundle") pod "9a2e37d4-2f38-466e-997b-b0ea9382fbb1" (UID: "9a2e37d4-2f38-466e-997b-b0ea9382fbb1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:03.428187 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.428160 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z" (OuterVolumeSpecName: "kube-api-access-zqx5z") pod "9a2e37d4-2f38-466e-997b-b0ea9382fbb1" (UID: "9a2e37d4-2f38-466e-997b-b0ea9382fbb1"). InnerVolumeSpecName "kube-api-access-zqx5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:58:03.428386 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.428365 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft" (OuterVolumeSpecName: "kube-api-access-z2kft") pod "cf7b3505-7b06-4171-8e1d-5c89caea98b3" (UID: "cf7b3505-7b06-4171-8e1d-5c89caea98b3"). InnerVolumeSpecName "kube-api-access-z2kft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:58:03.432151 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.432116 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util" (OuterVolumeSpecName: "util") pod "cf7b3505-7b06-4171-8e1d-5c89caea98b3" (UID: "cf7b3505-7b06-4171-8e1d-5c89caea98b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:03.526960 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.526935 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2kft\" (UniqueName: \"kubernetes.io/projected/cf7b3505-7b06-4171-8e1d-5c89caea98b3-kube-api-access-z2kft\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:03.526960 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.526958 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:03.527102 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.526967 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7b3505-7b06-4171-8e1d-5c89caea98b3-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:03.527102 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.526976 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqx5z\" (UniqueName: \"kubernetes.io/projected/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-kube-api-access-zqx5z\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:03.527102 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.526986 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:03.564443 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.564412 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util" (OuterVolumeSpecName: "util") pod "9a2e37d4-2f38-466e-997b-b0ea9382fbb1" (UID: "9a2e37d4-2f38-466e-997b-b0ea9382fbb1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 15:58:03.627331 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:03.627306 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a2e37d4-2f38-466e-997b-b0ea9382fbb1-util\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:04.182278 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.182253 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" Jul 08 15:58:04.182447 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.182252 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d" event={"ID":"9a2e37d4-2f38-466e-997b-b0ea9382fbb1","Type":"ContainerDied","Data":"1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5"} Jul 08 15:58:04.182447 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.182383 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a58d7450daa27f0e1e4bb37d9d2fa64675a5b4050d1a399ae33ab28690d12e5" Jul 08 15:58:04.184170 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.184142 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" Jul 08 15:58:04.184288 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.184143 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8" event={"ID":"cf7b3505-7b06-4171-8e1d-5c89caea98b3","Type":"ContainerDied","Data":"795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b"} Jul 08 15:58:04.184288 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:04.184258 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="795dda0e62490ff762b3a6d2e2516baf5a8779a2631f85e29d7884f73575d15b" Jul 08 15:58:11.930750 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:11.930714 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:58:36.961117 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:36.961056 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6748fdd9dd-7vz7h" podUID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" containerName="console" containerID="cri-o://7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57" gracePeriod=15 Jul 08 15:58:37.198716 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.198689 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:58:37.278024 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.277967 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278024 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278000 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278203 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278046 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278203 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278075 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278203 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278107 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278355 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278312 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278414 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278375 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlvqn\" (UniqueName: \"kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn\") pod \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\" (UID: \"044e10e3-3e8f-47d4-9c4b-cc217ef9396c\") " Jul 08 15:58:37.278477 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278448 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config" (OuterVolumeSpecName: "console-config") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:58:37.278531 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278455 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca" (OuterVolumeSpecName: "service-ca") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:58:37.278531 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278477 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:58:37.278716 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278696 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-trusted-ca-bundle\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.278780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278712 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 15:58:37.278780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278722 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.278780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.278760 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-config\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.280330 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.280298 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:58:37.280439 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.280321 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 15:58:37.280504 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.280487 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn" (OuterVolumeSpecName: "kube-api-access-mlvqn") pod "044e10e3-3e8f-47d4-9c4b-cc217ef9396c" (UID: "044e10e3-3e8f-47d4-9c4b-cc217ef9396c"). InnerVolumeSpecName "kube-api-access-mlvqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 15:58:37.339757 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.339730 2566 generic.go:356] "Generic (PLEG): container finished" podID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" containerID="7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57" exitCode=0 Jul 08 15:58:37.339859 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.339778 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6748fdd9dd-7vz7h" Jul 08 15:58:37.339859 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.339818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6748fdd9dd-7vz7h" event={"ID":"044e10e3-3e8f-47d4-9c4b-cc217ef9396c","Type":"ContainerDied","Data":"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57"} Jul 08 15:58:37.339984 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.339873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6748fdd9dd-7vz7h" event={"ID":"044e10e3-3e8f-47d4-9c4b-cc217ef9396c","Type":"ContainerDied","Data":"e99aea8f83741c64eb59dbe8268d2d6bb096102115d4152ef4ba078440b3fbe9"} Jul 08 15:58:37.339984 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.339902 2566 scope.go:117] "RemoveContainer" containerID="7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57" Jul 08 15:58:37.348601 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.348584 2566 scope.go:117] "RemoveContainer" containerID="7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57" Jul 08 15:58:37.348819 ip-10-0-136-198 kubenswrapper[2566]: E0708 15:58:37.348803 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57\": container with ID starting with 7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57 not found: ID does not exist" containerID="7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57" Jul 08 15:58:37.348873 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.348826 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57"} err="failed to get container status \"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57\": rpc error: code = NotFound desc = could not find container \"7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57\": container with ID starting with 7fe145e100b559410bac4cc5d0f845a47525daba766a395eab87a5494fac8d57 not found: ID does not exist" Jul 08 15:58:37.371780 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.371758 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:58:37.377096 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.377072 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6748fdd9dd-7vz7h"] Jul 08 15:58:37.379129 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.379113 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlvqn\" (UniqueName: \"kubernetes.io/projected/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-kube-api-access-mlvqn\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.379183 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.379132 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-oauth-config\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.379183 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.379147 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-console-serving-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:37.379183 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:37.379159 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/044e10e3-3e8f-47d4-9c4b-cc217ef9396c-oauth-serving-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 15:58:39.271580 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:39.271527 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" path="/var/lib/kubelet/pods/044e10e3-3e8f-47d4-9c4b-cc217ef9396c/volumes" Jul 08 15:58:41.879280 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879247 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr"] Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879613 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="extract" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879626 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="extract" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879640 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="util" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879649 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="util" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879660 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="pull" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879666 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="pull" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879675 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="pull" Jul 08 15:58:41.879682 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879681 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="pull" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879689 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="pull" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879694 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="pull" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879699 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="pull" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879704 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="pull" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879715 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" containerName="console" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879724 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" containerName="console" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879733 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879738 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879744 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879749 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879759 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879764 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879769 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879773 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879779 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879784 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879790 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879797 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="util" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879850 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9f0b268-389a-437c-9ea6-29e7750b9553" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879858 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="044e10e3-3e8f-47d4-9c4b-cc217ef9396c" containerName="console" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879863 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="027d33d0-e0d8-475b-948a-c31fc3bb95b7" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879869 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7b3505-7b06-4171-8e1d-5c89caea98b3" containerName="extract" Jul 08 15:58:41.879923 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.879874 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a2e37d4-2f38-466e-997b-b0ea9382fbb1" containerName="extract" Jul 08 15:58:41.882619 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.882597 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:41.885178 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.885155 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-n64pz\"" Jul 08 15:58:41.885279 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.885259 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 15:58:41.885845 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.885824 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 15:58:41.905214 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.905193 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr"] Jul 08 15:58:41.913399 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:41.913375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7jf\" (UniqueName: \"kubernetes.io/projected/218f8f58-d8be-4693-ad0a-145d7d5a26da-kube-api-access-hn7jf\") pod \"limitador-operator-controller-manager-859d7b78bd-rhbgr\" (UID: \"218f8f58-d8be-4693-ad0a-145d7d5a26da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:42.014504 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:42.014472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7jf\" (UniqueName: \"kubernetes.io/projected/218f8f58-d8be-4693-ad0a-145d7d5a26da-kube-api-access-hn7jf\") pod \"limitador-operator-controller-manager-859d7b78bd-rhbgr\" (UID: \"218f8f58-d8be-4693-ad0a-145d7d5a26da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:42.025929 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:42.025904 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7jf\" (UniqueName: \"kubernetes.io/projected/218f8f58-d8be-4693-ad0a-145d7d5a26da-kube-api-access-hn7jf\") pod \"limitador-operator-controller-manager-859d7b78bd-rhbgr\" (UID: \"218f8f58-d8be-4693-ad0a-145d7d5a26da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:42.193725 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:42.193664 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:42.331974 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:42.331917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr"] Jul 08 15:58:42.335240 ip-10-0-136-198 kubenswrapper[2566]: W0708 15:58:42.335214 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218f8f58_d8be_4693_ad0a_145d7d5a26da.slice/crio-22fe47268e59f4da014492007f9c7693d9fc5de7c62e005d8be8b6a5406be9d1 WatchSource:0}: Error finding container 22fe47268e59f4da014492007f9c7693d9fc5de7c62e005d8be8b6a5406be9d1: Status 404 returned error can't find the container with id 22fe47268e59f4da014492007f9c7693d9fc5de7c62e005d8be8b6a5406be9d1 Jul 08 15:58:42.365620 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:42.365596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" event={"ID":"218f8f58-d8be-4693-ad0a-145d7d5a26da","Type":"ContainerStarted","Data":"22fe47268e59f4da014492007f9c7693d9fc5de7c62e005d8be8b6a5406be9d1"} Jul 08 15:58:45.382249 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:45.382216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" event={"ID":"218f8f58-d8be-4693-ad0a-145d7d5a26da","Type":"ContainerStarted","Data":"d2d7386a77398f79c3d13907ef03a96d4cafeab9c181d841ac3a415c8e0e4eed"} Jul 08 15:58:45.382727 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:45.382268 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 15:58:45.402276 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:45.402232 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" podStartSLOduration=2.212904856 podStartE2EDuration="4.402219287s" totalImagesPullingTime="2.189314431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 15:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 15:58:45.402010412 +0000 UTC m=+470.665029094" watchObservedRunningTime="2026-07-08 15:58:45.402219287 +0000 UTC m=+470.665237955" Jul 08 15:58:56.389283 ip-10-0-136-198 kubenswrapper[2566]: I0708 15:58:56.389246 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rhbgr" Jul 08 16:00:55.162544 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:00:55.162514 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:00:55.165349 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:00:55.165327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:00:55.168687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:00:55.168667 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:00:55.170826 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:00:55.170808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:01:01.183552 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.183517 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256"] Jul 08 16:01:01.187168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.187151 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.190671 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.190648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 16:01:01.190671 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.190656 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gjc5h\"" Jul 08 16:01:01.190842 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.190686 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 16:01:01.190842 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.190706 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 16:01:01.203466 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.203442 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256"] Jul 08 16:01:01.274356 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.274532 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.274532 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.274532 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.274532 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2dp5\" (UniqueName: \"kubernetes.io/projected/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-kube-api-access-g2dp5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.274719 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.274619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.375719 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.375683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2dp5\" (UniqueName: \"kubernetes.io/projected/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-kube-api-access-g2dp5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.375889 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.375744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.375889 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.375817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.375889 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.375847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.376126 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.376082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.376261 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.376174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.376338 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.376276 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.376421 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.376398 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.376514 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.376428 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.378504 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.378476 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.378593 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.378500 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.387451 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.387428 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2dp5\" (UniqueName: \"kubernetes.io/projected/a4c1b4f6-e94c-45d6-a508-d0e8fa3befea-kube-api-access-g2dp5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256\" (UID: \"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.497342 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.497253 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:01.630313 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:01.630273 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256"] Jul 08 16:01:01.631647 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:01:01.631621 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c1b4f6_e94c_45d6_a508_d0e8fa3befea.slice/crio-68cd83d8a4d1bba5a0b1edef2c3339bb030cb89fdd250fa7efc9a59e1ba339f2 WatchSource:0}: Error finding container 68cd83d8a4d1bba5a0b1edef2c3339bb030cb89fdd250fa7efc9a59e1ba339f2: Status 404 returned error can't find the container with id 68cd83d8a4d1bba5a0b1edef2c3339bb030cb89fdd250fa7efc9a59e1ba339f2 Jul 08 16:01:02.033530 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:02.033494 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" event={"ID":"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea","Type":"ContainerStarted","Data":"68cd83d8a4d1bba5a0b1edef2c3339bb030cb89fdd250fa7efc9a59e1ba339f2"} Jul 08 16:01:09.071665 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:09.071620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" event={"ID":"a4c1b4f6-e94c-45d6-a508-d0e8fa3befea","Type":"ContainerStarted","Data":"3f57b85a54f1050ba3feb1a9b4a41126358c7e19acc5e52d5adeab1311cb1dc6"} Jul 08 16:01:09.092859 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:09.092811 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" podStartSLOduration=1.132172711 podStartE2EDuration="8.092796329s" totalImagesPullingTime="6.960623618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:01:09.09153731 +0000 UTC m=+614.354555981" watchObservedRunningTime="2026-07-08 16:01:09.092796329 +0000 UTC m=+614.355814998" Jul 08 16:01:11.498118 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:11.498078 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:11.498512 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:11.498141 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:11.510793 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:11.510766 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:12.097754 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:12.097725 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256" Jul 08 16:01:53.384333 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.384306 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9"] Jul 08 16:01:53.386614 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.386598 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.388850 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.388827 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 16:01:53.397934 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.397913 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9"] Jul 08 16:01:53.473778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da273f17-1924-4f7f-b612-180da4ee9fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.473919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.473919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggrwc\" (UniqueName: \"kubernetes.io/projected/da273f17-1924-4f7f-b612-180da4ee9fd1-kube-api-access-ggrwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.473919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.473919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473911 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.474065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.473929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574283 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggrwc\" (UniqueName: \"kubernetes.io/projected/da273f17-1924-4f7f-b612-180da4ee9fd1-kube-api-access-ggrwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574283 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574498 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574305 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574498 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574498 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574355 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da273f17-1924-4f7f-b612-180da4ee9fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574498 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574770 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574744 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574770 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574764 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.574875 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.574805 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.576843 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.576819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da273f17-1924-4f7f-b612-180da4ee9fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.576990 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.576973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da273f17-1924-4f7f-b612-180da4ee9fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.581962 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.581937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggrwc\" (UniqueName: \"kubernetes.io/projected/da273f17-1924-4f7f-b612-180da4ee9fd1-kube-api-access-ggrwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-56mq9\" (UID: \"da273f17-1924-4f7f-b612-180da4ee9fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.697100 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.697033 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:01:53.831449 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.831420 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9"] Jul 08 16:01:53.832926 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:01:53.832903 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda273f17_1924_4f7f_b612_180da4ee9fd1.slice/crio-de2c42bc739a6a9803e0c7e1c5a5250db09a3fb270b11ae266c1fce98fdc06bc WatchSource:0}: Error finding container de2c42bc739a6a9803e0c7e1c5a5250db09a3fb270b11ae266c1fce98fdc06bc: Status 404 returned error can't find the container with id de2c42bc739a6a9803e0c7e1c5a5250db09a3fb270b11ae266c1fce98fdc06bc Jul 08 16:01:53.834251 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:53.834230 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:01:54.286894 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:54.286862 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" event={"ID":"da273f17-1924-4f7f-b612-180da4ee9fd1","Type":"ContainerStarted","Data":"a52c98f9b465bb9399a9369c3f49c686019e9ba99ceb7d7361b0d00bcbcd3e50"} Jul 08 16:01:54.287057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:54.286911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" event={"ID":"da273f17-1924-4f7f-b612-180da4ee9fd1","Type":"ContainerStarted","Data":"de2c42bc739a6a9803e0c7e1c5a5250db09a3fb270b11ae266c1fce98fdc06bc"} Jul 08 16:01:54.307847 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:01:54.307796 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" podStartSLOduration=1.042967497 podStartE2EDuration="1.307782835s" totalImagesPullingTime="264.815338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:01:54.306151083 +0000 UTC m=+659.569169752" watchObservedRunningTime="2026-07-08 16:01:54.307782835 +0000 UTC m=+659.570801504" Jul 08 16:02:03.698121 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:02:03.698088 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:02:03.698591 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:02:03.698136 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:02:03.710692 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:02:03.710668 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:02:04.344702 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:02:04.344665 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-56mq9" Jul 08 16:05:55.195046 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:05:55.194963 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:05:55.198496 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:05:55.198473 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:05:55.200958 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:05:55.200930 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:05:55.203903 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:05:55.203884 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:10:55.229419 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:10:55.229388 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:10:55.234025 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:10:55.234003 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:10:55.234546 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:10:55.234525 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:10:55.239069 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:10:55.239054 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:15:55.260528 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:15:55.260504 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:15:55.267532 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:15:55.267508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:15:55.268015 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:15:55.267989 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:15:55.273319 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:15:55.273303 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:16:56.129831 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.129799 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:16:56.132973 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.132956 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.135456 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.135435 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:16:56.135646 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.135632 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6820fd9e-serving-cert\"" Jul 08 16:16:56.136125 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.136109 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:16:56.136200 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.136120 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:16:56.144042 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.144020 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:16:56.238291 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.238260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.238433 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.238297 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.238433 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.238406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptn7c\" (UniqueName: \"kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.339274 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.339244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.339274 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.339274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.339438 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.339308 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptn7c\" (UniqueName: \"kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.339909 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.339887 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.341807 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.341787 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.347063 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.347044 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptn7c\" (UniqueName: \"kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c\") pod \"maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.443462 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.443395 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:56.576752 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.576726 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:16:56.578507 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:16:56.578478 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c826ba3_b923_4f83_931e_ec5cb486abb6.slice/crio-6cafed77c0e27af5dc685725f39674dc61b8c3c3ee3a2cc0dff92c723fad05e1 WatchSource:0}: Error finding container 6cafed77c0e27af5dc685725f39674dc61b8c3c3ee3a2cc0dff92c723fad05e1: Status 404 returned error can't find the container with id 6cafed77c0e27af5dc685725f39674dc61b8c3c3ee3a2cc0dff92c723fad05e1 Jul 08 16:16:56.580252 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.580237 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:16:56.712431 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:56.712351 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" event={"ID":"9c826ba3-b923-4f83-931e-ec5cb486abb6","Type":"ContainerStarted","Data":"6cafed77c0e27af5dc685725f39674dc61b8c3c3ee3a2cc0dff92c723fad05e1"} Jul 08 16:16:59.731354 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:59.731319 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" event={"ID":"9c826ba3-b923-4f83-931e-ec5cb486abb6","Type":"ContainerStarted","Data":"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af"} Jul 08 16:16:59.731850 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:59.731448 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:16:59.750416 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:16:59.750370 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" podStartSLOduration=1.5988683940000001 podStartE2EDuration="3.750356488s" totalImagesPullingTime="2.151488094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:16:59.748302713 +0000 UTC m=+1565.011321383" watchObservedRunningTime="2026-07-08 16:16:59.750356488 +0000 UTC m=+1565.013375157" Jul 08 16:17:01.556355 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:01.556323 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:17:01.741447 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:01.741404 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" podUID="9c826ba3-b923-4f83-931e-ec5cb486abb6" containerName="maas-api" containerID="cri-o://05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af" gracePeriod=30 Jul 08 16:17:01.990346 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:01.990318 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:17:02.087588 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.087533 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls\") pod \"9c826ba3-b923-4f83-931e-ec5cb486abb6\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " Jul 08 16:17:02.087745 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.087690 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca\") pod \"9c826ba3-b923-4f83-931e-ec5cb486abb6\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " Jul 08 16:17:02.087745 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.087718 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptn7c\" (UniqueName: \"kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c\") pod \"9c826ba3-b923-4f83-931e-ec5cb486abb6\" (UID: \"9c826ba3-b923-4f83-931e-ec5cb486abb6\") " Jul 08 16:17:02.088081 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.088047 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c826ba3-b923-4f83-931e-ec5cb486abb6" (UID: "9c826ba3-b923-4f83-931e-ec5cb486abb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:17:02.089756 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.089722 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c826ba3-b923-4f83-931e-ec5cb486abb6" (UID: "9c826ba3-b923-4f83-931e-ec5cb486abb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:17:02.089965 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.089949 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c" (OuterVolumeSpecName: "kube-api-access-ptn7c") pod "9c826ba3-b923-4f83-931e-ec5cb486abb6" (UID: "9c826ba3-b923-4f83-931e-ec5cb486abb6"). InnerVolumeSpecName "kube-api-access-ptn7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:17:02.189110 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.189054 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c826ba3-b923-4f83-931e-ec5cb486abb6-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:02.189110 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.189075 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptn7c\" (UniqueName: \"kubernetes.io/projected/9c826ba3-b923-4f83-931e-ec5cb486abb6-kube-api-access-ptn7c\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:02.189110 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.189085 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c826ba3-b923-4f83-931e-ec5cb486abb6-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:02.748092 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.748059 2566 generic.go:356] "Generic (PLEG): container finished" podID="9c826ba3-b923-4f83-931e-ec5cb486abb6" containerID="05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af" exitCode=0 Jul 08 16:17:02.748486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.748115 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" Jul 08 16:17:02.748486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.748159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" event={"ID":"9c826ba3-b923-4f83-931e-ec5cb486abb6","Type":"ContainerDied","Data":"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af"} Jul 08 16:17:02.748486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.748229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg" event={"ID":"9c826ba3-b923-4f83-931e-ec5cb486abb6","Type":"ContainerDied","Data":"6cafed77c0e27af5dc685725f39674dc61b8c3c3ee3a2cc0dff92c723fad05e1"} Jul 08 16:17:02.748486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.748255 2566 scope.go:117] "RemoveContainer" containerID="05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af" Jul 08 16:17:02.759632 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.759610 2566 scope.go:117] "RemoveContainer" containerID="05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af" Jul 08 16:17:02.759899 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:17:02.759879 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af\": container with ID starting with 05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af not found: ID does not exist" containerID="05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af" Jul 08 16:17:02.759976 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.759906 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af"} err="failed to get container status \"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af\": rpc error: code = NotFound desc = could not find container \"05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af\": container with ID starting with 05bdb0183bfc1523bebc382ec0033b732ef6044ae2c9710f088dd5e9e6b117af not found: ID does not exist" Jul 08 16:17:02.774069 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.774040 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:17:02.776765 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:02.776746 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6820fd9e-5885bfb67f-jx7zg"] Jul 08 16:17:03.269597 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:03.269540 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c826ba3-b923-4f83-931e-ec5cb486abb6" path="/var/lib/kubelet/pods/9c826ba3-b923-4f83-931e-ec5cb486abb6/volumes" Jul 08 16:17:07.511706 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.511671 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:07.512078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.511988 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c826ba3-b923-4f83-931e-ec5cb486abb6" containerName="maas-api" Jul 08 16:17:07.512078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.512000 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c826ba3-b923-4f83-931e-ec5cb486abb6" containerName="maas-api" Jul 08 16:17:07.512078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.512063 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c826ba3-b923-4f83-931e-ec5cb486abb6" containerName="maas-api" Jul 08 16:17:07.516111 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.516090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.518222 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.518159 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-36118105-openshift-default-dockercfg-dflgt\"" Jul 08 16:17:07.528307 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.528278 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:07.636242 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636242 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636464 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636464 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636464 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636464 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b97mw\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.636690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.636627 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737430 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737430 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737489 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737489 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737561 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737650 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b97mw\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737701 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737701 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737833 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737811 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737921 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737901 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.737983 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.737967 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.738715 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.738691 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.739933 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.739909 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.740059 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.740042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.746924 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.746905 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.748685 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.748662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b97mw\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw\") pod \"e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.827193 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.827131 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:07.966013 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:07.965989 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:07.967511 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:17:07.967480 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b79c8ee_8639_4e89_9a54_9e2f8fa2b088.slice/crio-60f2c498059efb9f400cf54a5e37cc0bb447b433206590852fa1a4e9dec64628 WatchSource:0}: Error finding container 60f2c498059efb9f400cf54a5e37cc0bb447b433206590852fa1a4e9dec64628: Status 404 returned error can't find the container with id 60f2c498059efb9f400cf54a5e37cc0bb447b433206590852fa1a4e9dec64628 Jul 08 16:17:08.780255 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:08.780224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" event={"ID":"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088","Type":"ContainerStarted","Data":"60f2c498059efb9f400cf54a5e37cc0bb447b433206590852fa1a4e9dec64628"} Jul 08 16:17:10.802379 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:10.802325 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:17:10.802707 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:10.802405 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:17:10.802707 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:10.802438 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:17:11.798815 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:11.798781 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" event={"ID":"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088","Type":"ContainerStarted","Data":"7ab8d4af1f02f5f678ca8a8b6a2bb427c4a0ffd6e8870168e80ef9932d1101e3"} Jul 08 16:17:11.821563 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:11.821517 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" podStartSLOduration=1.988865219 podStartE2EDuration="4.821501757s" totalImagesPullingTime="2.832636538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:17:11.820260792 +0000 UTC m=+1577.083279472" watchObservedRunningTime="2026-07-08 16:17:11.821501757 +0000 UTC m=+1577.084520426" Jul 08 16:17:11.828316 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:11.828287 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:11.832902 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:11.832882 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:12.803857 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:12.803827 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:12.804713 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:12.804692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:19.844549 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:19.844520 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:19.844885 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:19.844766 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" podUID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" containerName="istio-proxy" containerID="cri-o://7ab8d4af1f02f5f678ca8a8b6a2bb427c4a0ffd6e8870168e80ef9932d1101e3" gracePeriod=30 Jul 08 16:17:19.867028 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:19.866999 2566 generic.go:356] "Generic (PLEG): container finished" podID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" containerID="7ab8d4af1f02f5f678ca8a8b6a2bb427c4a0ffd6e8870168e80ef9932d1101e3" exitCode=0 Jul 08 16:17:19.867172 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:19.867098 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" event={"ID":"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088","Type":"ContainerDied","Data":"7ab8d4af1f02f5f678ca8a8b6a2bb427c4a0ffd6e8870168e80ef9932d1101e3"} Jul 08 16:17:19.987176 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:19.987123 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:20.047652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047622 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.047797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.047797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047687 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.047797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047727 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b97mw\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.047797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047762 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.047797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047785 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.048072 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047809 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.048072 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047852 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.048072 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.047874 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert\") pod \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\" (UID: \"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088\") " Jul 08 16:17:20.048495 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.048452 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:17:20.048637 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.048476 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:17:20.048637 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.048545 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:17:20.048637 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.048621 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:17:20.048838 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.048751 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:17:20.050634 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.050448 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw" (OuterVolumeSpecName: "kube-api-access-b97mw") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "kube-api-access-b97mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:17:20.050634 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.050500 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:17:20.050784 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.050631 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:17:20.050961 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.050934 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" (UID: "5b79c8ee-8639-4e89-9a54-9e2f8fa2b088"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:17:20.148867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148837 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.148867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148863 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.148867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148873 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148882 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148891 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148899 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148907 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148914 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.149065 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.148921 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b97mw\" (UniqueName: \"kubernetes.io/projected/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088-kube-api-access-b97mw\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:20.873048 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.873008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" event={"ID":"5b79c8ee-8639-4e89-9a54-9e2f8fa2b088","Type":"ContainerDied","Data":"60f2c498059efb9f400cf54a5e37cc0bb447b433206590852fa1a4e9dec64628"} Jul 08 16:17:20.873048 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.873041 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq" Jul 08 16:17:20.873525 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.873076 2566 scope.go:117] "RemoveContainer" containerID="7ab8d4af1f02f5f678ca8a8b6a2bb427c4a0ffd6e8870168e80ef9932d1101e3" Jul 08 16:17:20.895116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.895092 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:20.898925 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:20.898905 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-36118105-openshift-default-9bd5bcc45-96bgq"] Jul 08 16:17:21.269082 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:21.269012 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" path="/var/lib/kubelet/pods/5b79c8ee-8639-4e89-9a54-9e2f8fa2b088/volumes" Jul 08 16:17:27.985286 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.985253 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:27.985814 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.985746 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" containerName="istio-proxy" Jul 08 16:17:27.985814 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.985765 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" containerName="istio-proxy" Jul 08 16:17:27.985928 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.985871 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b79c8ee-8639-4e89-9a54-9e2f8fa2b088" containerName="istio-proxy" Jul 08 16:17:27.990535 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.990513 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:27.992777 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.992758 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:17:27.993369 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.993357 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:17:27.993434 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.993350 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:17:27.993434 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:27.993397 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-326e35c8-serving-cert\"" Jul 08 16:17:28.000201 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.000175 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:28.112866 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.112829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.113034 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.112874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cljx9\" (UniqueName: \"kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.113034 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.112939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.213545 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.213508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.213715 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.213562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cljx9\" (UniqueName: \"kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.213715 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.213638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.213715 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:17:28.213674 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-serving-cert: secret "maas-api-e2e-derive-326e35c8-serving-cert" not found Jul 08 16:17:28.213867 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:17:28.213731 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls podName:26cbc350-6ee7-494c-9170-597824683f4b nodeName:}" failed. No retries permitted until 2026-07-08 16:17:28.713716234 +0000 UTC m=+1593.976734881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls") pod "maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" (UID: "26cbc350-6ee7-494c-9170-597824683f4b") : secret "maas-api-e2e-derive-326e35c8-serving-cert" not found Jul 08 16:17:28.214271 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.214250 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.224017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.223987 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cljx9\" (UniqueName: \"kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.717470 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.717434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.719976 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.719948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") pod \"maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:28.901386 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:28.901354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:29.052057 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:17:29.052017 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26cbc350_6ee7_494c_9170_597824683f4b.slice/crio-6b5fc5b49758e14776f4707fa5be61bf9e47852c80734d04aefb03a061ed4c69 WatchSource:0}: Error finding container 6b5fc5b49758e14776f4707fa5be61bf9e47852c80734d04aefb03a061ed4c69: Status 404 returned error can't find the container with id 6b5fc5b49758e14776f4707fa5be61bf9e47852c80734d04aefb03a061ed4c69 Jul 08 16:17:29.052913 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:29.052878 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:29.922505 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:29.922468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" event={"ID":"26cbc350-6ee7-494c-9170-597824683f4b","Type":"ContainerStarted","Data":"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e"} Jul 08 16:17:29.922919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:29.922681 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:29.922919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:29.922727 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" event={"ID":"26cbc350-6ee7-494c-9170-597824683f4b","Type":"ContainerStarted","Data":"6b5fc5b49758e14776f4707fa5be61bf9e47852c80734d04aefb03a061ed4c69"} Jul 08 16:17:29.945020 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:29.944968 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" podStartSLOduration=2.58910076 podStartE2EDuration="2.944953351s" totalImagesPullingTime="355.852591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:17:29.94346659 +0000 UTC m=+1595.206485453" watchObservedRunningTime="2026-07-08 16:17:29.944953351 +0000 UTC m=+1595.207972026" Jul 08 16:17:32.425380 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.425349 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:32.425848 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.425556 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" podUID="26cbc350-6ee7-494c-9170-597824683f4b" containerName="maas-api" containerID="cri-o://6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e" gracePeriod=30 Jul 08 16:17:32.661404 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.661378 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:32.749527 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.749460 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") pod \"26cbc350-6ee7-494c-9170-597824683f4b\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " Jul 08 16:17:32.749527 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.749489 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca\") pod \"26cbc350-6ee7-494c-9170-597824683f4b\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " Jul 08 16:17:32.749728 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.749641 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cljx9\" (UniqueName: \"kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9\") pod \"26cbc350-6ee7-494c-9170-597824683f4b\" (UID: \"26cbc350-6ee7-494c-9170-597824683f4b\") " Jul 08 16:17:32.749889 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.749866 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26cbc350-6ee7-494c-9170-597824683f4b" (UID: "26cbc350-6ee7-494c-9170-597824683f4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:17:32.751761 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.751738 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26cbc350-6ee7-494c-9170-597824683f4b" (UID: "26cbc350-6ee7-494c-9170-597824683f4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:17:32.751761 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.751737 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9" (OuterVolumeSpecName: "kube-api-access-cljx9") pod "26cbc350-6ee7-494c-9170-597824683f4b" (UID: "26cbc350-6ee7-494c-9170-597824683f4b"). InnerVolumeSpecName "kube-api-access-cljx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:17:32.850998 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.850972 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26cbc350-6ee7-494c-9170-597824683f4b-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:32.850998 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.850999 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26cbc350-6ee7-494c-9170-597824683f4b-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:32.851146 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.851008 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cljx9\" (UniqueName: \"kubernetes.io/projected/26cbc350-6ee7-494c-9170-597824683f4b-kube-api-access-cljx9\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:17:32.938942 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.938910 2566 generic.go:356] "Generic (PLEG): container finished" podID="26cbc350-6ee7-494c-9170-597824683f4b" containerID="6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e" exitCode=0 Jul 08 16:17:32.939078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.938957 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" Jul 08 16:17:32.939078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.938968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" event={"ID":"26cbc350-6ee7-494c-9170-597824683f4b","Type":"ContainerDied","Data":"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e"} Jul 08 16:17:32.939078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.939019 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl" event={"ID":"26cbc350-6ee7-494c-9170-597824683f4b","Type":"ContainerDied","Data":"6b5fc5b49758e14776f4707fa5be61bf9e47852c80734d04aefb03a061ed4c69"} Jul 08 16:17:32.939078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.939050 2566 scope.go:117] "RemoveContainer" containerID="6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e" Jul 08 16:17:32.948690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.948658 2566 scope.go:117] "RemoveContainer" containerID="6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e" Jul 08 16:17:32.948966 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:17:32.948944 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e\": container with ID starting with 6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e not found: ID does not exist" containerID="6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e" Jul 08 16:17:32.949023 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.948976 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e"} err="failed to get container status \"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e\": rpc error: code = NotFound desc = could not find container \"6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e\": container with ID starting with 6e0b05b80ad19abe3dd602696599bf9aaaf09756ecae322b48c60bb48bc1e89e not found: ID does not exist" Jul 08 16:17:32.960370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.960347 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:32.964197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:32.964179 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-326e35c8-cdbf99d67-4hffl"] Jul 08 16:17:33.268644 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:17:33.268616 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26cbc350-6ee7-494c-9170-597824683f4b" path="/var/lib/kubelet/pods/26cbc350-6ee7-494c-9170-597824683f4b/volumes" Jul 08 16:19:15.940164 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.940132 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:15.940663 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.940520 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26cbc350-6ee7-494c-9170-597824683f4b" containerName="maas-api" Jul 08 16:19:15.940663 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.940534 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="26cbc350-6ee7-494c-9170-597824683f4b" containerName="maas-api" Jul 08 16:19:15.940663 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.940606 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="26cbc350-6ee7-494c-9170-597824683f4b" containerName="maas-api" Jul 08 16:19:15.942413 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.942397 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:15.944521 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.944499 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-74123113-openshift-default-dockercfg-qvf4j\"" Jul 08 16:19:15.970178 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:15.970149 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:16.047117 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047117 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbn58\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047296 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047460 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047460 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.047563 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.047478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148322 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148322 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbn58\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148831 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148537 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148891 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148939 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148919 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.148976 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.148950 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.149116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.149093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.151012 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.149757 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.151713 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.151688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.154803 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.154781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.157296 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.157275 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.158058 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.157946 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbn58\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58\") pod \"e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.251805 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.251746 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:16.386298 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.386274 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:16.387879 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:19:16.387846 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895e2cb8_0c7e_4427_8f5a_5b43b7b6f7ee.slice/crio-ad782204605701f059ad71bfc9ebc73c9496528e0488e1fc2d0c58401f635fb9 WatchSource:0}: Error finding container ad782204605701f059ad71bfc9ebc73c9496528e0488e1fc2d0c58401f635fb9: Status 404 returned error can't find the container with id ad782204605701f059ad71bfc9ebc73c9496528e0488e1fc2d0c58401f635fb9 Jul 08 16:19:16.389955 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.389924 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:16.390044 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.389984 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:16.390044 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.390012 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:16.446600 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.446550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerStarted","Data":"ac70bdcb65cce965d16f3550bde723d5f0f322e28f2b86b1d2998c0211e6f801"} Jul 08 16:19:16.446728 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.446611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerStarted","Data":"ad782204605701f059ad71bfc9ebc73c9496528e0488e1fc2d0c58401f635fb9"} Jul 08 16:19:16.468493 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:16.468442 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podStartSLOduration=1.468422128 podStartE2EDuration="1.468422128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:19:16.466829288 +0000 UTC m=+1701.729847957" watchObservedRunningTime="2026-07-08 16:19:16.468422128 +0000 UTC m=+1701.731440798" Jul 08 16:19:17.252136 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:17.252097 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:17.256739 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:17.256716 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:17.450894 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:17.450864 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:17.451679 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:17.451661 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:32.452911 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.452874 2566 patch_prober.go:28] interesting pod/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 16:19:32.453306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.452933 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 16:19:32.532984 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.532953 2566 generic.go:356] "Generic (PLEG): container finished" podID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerID="ac70bdcb65cce965d16f3550bde723d5f0f322e28f2b86b1d2998c0211e6f801" exitCode=0 Jul 08 16:19:32.533138 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.532993 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerDied","Data":"ac70bdcb65cce965d16f3550bde723d5f0f322e28f2b86b1d2998c0211e6f801"} Jul 08 16:19:32.533332 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.533319 2566 scope.go:117] "RemoveContainer" containerID="ac70bdcb65cce965d16f3550bde723d5f0f322e28f2b86b1d2998c0211e6f801" Jul 08 16:19:32.534270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.534242 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:32.534323 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.534311 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:32.534363 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:32.534351 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:33.541455 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:33.541415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerStarted","Data":"43739df85bdefd7830a8613f26d2d062c6cb76f4ebd00af67ed67a5023a387b6"} Jul 08 16:19:34.252301 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:34.252260 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:34.252503 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:34.252481 2566 patch_prober.go:28] interesting pod/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 16:19:34.252549 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:34.252531 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 16:19:35.252912 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:35.252875 2566 patch_prober.go:28] interesting pod/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 16:19:35.253286 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:35.252943 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 16:19:36.253103 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:36.253066 2566 patch_prober.go:28] interesting pod/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 16:19:36.253597 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:36.253122 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 16:19:37.252900 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:37.252868 2566 patch_prober.go:28] interesting pod/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 16:19:37.253084 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:37.252921 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 16:19:37.318885 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:37.318845 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:37.319354 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:37.319074 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" containerID="cri-o://43739df85bdefd7830a8613f26d2d062c6cb76f4ebd00af67ed67a5023a387b6" gracePeriod=30 Jul 08 16:19:42.588879 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.588846 2566 generic.go:356] "Generic (PLEG): container finished" podID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerID="43739df85bdefd7830a8613f26d2d062c6cb76f4ebd00af67ed67a5023a387b6" exitCode=0 Jul 08 16:19:42.589270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.588896 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerDied","Data":"43739df85bdefd7830a8613f26d2d062c6cb76f4ebd00af67ed67a5023a387b6"} Jul 08 16:19:42.589270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.588943 2566 scope.go:117] "RemoveContainer" containerID="ac70bdcb65cce965d16f3550bde723d5f0f322e28f2b86b1d2998c0211e6f801" Jul 08 16:19:42.687405 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.687351 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:42.768779 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.768751 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.768923 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.768782 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.768923 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.768809 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.768923 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.768829 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbn58\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769094 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.768967 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769094 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769019 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769094 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769081 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769261 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769119 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769261 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769118 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:19:42.769261 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769145 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token\") pod \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\" (UID: \"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee\") " Jul 08 16:19:42.769411 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769245 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:19:42.769411 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769312 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:19:42.769716 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769509 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:19:42.769774 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769725 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.769774 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769750 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.769774 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.769766 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.770251 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.770224 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:19:42.771185 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.771157 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:19:42.771284 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.771200 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58" (OuterVolumeSpecName: "kube-api-access-pbn58") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "kube-api-access-pbn58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:19:42.771284 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.771237 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:19:42.771450 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.771435 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" (UID: "895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:19:42.871057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871014 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.871057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871052 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.871057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871062 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbn58\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-kube-api-access-pbn58\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.871057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871074 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.871292 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871083 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:42.871292 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:42.871090 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:19:43.594756 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:43.594673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" event={"ID":"895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee","Type":"ContainerDied","Data":"ad782204605701f059ad71bfc9ebc73c9496528e0488e1fc2d0c58401f635fb9"} Jul 08 16:19:43.594756 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:43.594728 2566 scope.go:117] "RemoveContainer" containerID="43739df85bdefd7830a8613f26d2d062c6cb76f4ebd00af67ed67a5023a387b6" Jul 08 16:19:43.595216 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:43.594685 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg" Jul 08 16:19:43.613115 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:43.613092 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:43.617122 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:43.617099 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-74123113-openshift-default-5cb4d44f5f-ks5bg"] Jul 08 16:19:45.268816 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:45.268783 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" path="/var/lib/kubelet/pods/895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee/volumes" Jul 08 16:19:57.598219 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598189 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:19:57.598638 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598526 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.598638 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598538 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.598638 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598557 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.598638 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598562 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.599033 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598645 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.599033 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.598655 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="895e2cb8-0c7e-4427-8f5a-5b43b7b6f7ee" containerName="istio-proxy" Jul 08 16:19:57.600435 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.600415 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.602979 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.602957 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-68603676-openshift-default-dockercfg-npcbl\"" Jul 08 16:19:57.616447 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.616424 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:19:57.681890 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.681865 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.681914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7hkd\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.681949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.681974 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.681996 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682017 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.682011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682273 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.682036 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682273 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.682142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.682273 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.682208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783247 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783372 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783372 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783308 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783372 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783520 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783471 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7hkd\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783520 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783644 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783644 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783644 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783827 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783803 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783912 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783888 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.783955 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.783938 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.784023 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.784004 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.784059 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.784031 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.785752 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.785733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.786354 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.786331 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.795148 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.795124 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7hkd\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.796813 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.796796 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token\") pod \"e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:57.910327 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:57.910303 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:58.047622 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.047596 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:19:58.049071 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:19:58.049045 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589f2fad_e5e9_4d92_b5d3_65da9ebb1c25.slice/crio-a76891ceb80c38e4ff81907149fd9c27d6df3c35c015aad918572edb23ce8ef4 WatchSource:0}: Error finding container a76891ceb80c38e4ff81907149fd9c27d6df3c35c015aad918572edb23ce8ef4: Status 404 returned error can't find the container with id a76891ceb80c38e4ff81907149fd9c27d6df3c35c015aad918572edb23ce8ef4 Jul 08 16:19:58.051175 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.051143 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:58.051248 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.051204 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:58.051248 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.051232 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:19:58.671881 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.671849 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerStarted","Data":"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b"} Jul 08 16:19:58.672306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.671925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerStarted","Data":"a76891ceb80c38e4ff81907149fd9c27d6df3c35c015aad918572edb23ce8ef4"} Jul 08 16:19:58.706117 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.706063 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podStartSLOduration=1.7060488249999999 podStartE2EDuration="1.706048825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:19:58.703779279 +0000 UTC m=+1743.966797947" watchObservedRunningTime="2026-07-08 16:19:58.706048825 +0000 UTC m=+1743.969067494" Jul 08 16:19:58.910756 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:58.910712 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:19:59.915364 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:19:59.915332 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:00.683294 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.683259 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:00.684172 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.684152 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:00.695561 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.695534 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:00.698693 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.698672 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.701269 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.701247 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-68603676-serving-cert\"" Jul 08 16:20:00.701867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.701851 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:20:00.702096 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.702078 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:20:00.702168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.702125 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:20:00.716599 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.716560 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:00.811365 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.811335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.811657 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.811391 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.811657 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.811443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4rdw\" (UniqueName: \"kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.912718 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.912688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.912910 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.912732 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.912910 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.912751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4rdw\" (UniqueName: \"kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.913626 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.913597 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.915348 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.915324 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:00.942186 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:00.942120 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4rdw\" (UniqueName: \"kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw\") pod \"maas-api-e2e-mt-68603676-56cff6d9dd-v8bss\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:01.009115 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:01.009080 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:01.151272 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:01.151249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:01.152994 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:20:01.152967 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922a760a_aa35_4ead_8104_7bf9b88bad37.slice/crio-f7bd1f3d7482a66a1fa67d9ee3612a29d9a22a097e44e1283187f07e3eab3dd2 WatchSource:0}: Error finding container f7bd1f3d7482a66a1fa67d9ee3612a29d9a22a097e44e1283187f07e3eab3dd2: Status 404 returned error can't find the container with id f7bd1f3d7482a66a1fa67d9ee3612a29d9a22a097e44e1283187f07e3eab3dd2 Jul 08 16:20:01.689499 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:01.689463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" event={"ID":"922a760a-aa35-4ead-8104-7bf9b88bad37","Type":"ContainerStarted","Data":"f7bd1f3d7482a66a1fa67d9ee3612a29d9a22a097e44e1283187f07e3eab3dd2"} Jul 08 16:20:02.699507 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:02.699459 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" event={"ID":"922a760a-aa35-4ead-8104-7bf9b88bad37","Type":"ContainerStarted","Data":"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063"} Jul 08 16:20:02.719844 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:02.719797 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" podStartSLOduration=1.829131776 podStartE2EDuration="2.719782764s" totalImagesPullingTime="890.650988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:20:02.717215546 +0000 UTC m=+1747.980234215" watchObservedRunningTime="2026-07-08 16:20:02.719782764 +0000 UTC m=+1747.982801434" Jul 08 16:20:03.704496 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:03.704459 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:09.722881 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:09.722849 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:13.775815 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.775723 2566 generic.go:356] "Generic (PLEG): container finished" podID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerID="bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b" exitCode=0 Jul 08 16:20:13.775815 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.775767 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerDied","Data":"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b"} Jul 08 16:20:13.776378 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.776256 2566 scope.go:117] "RemoveContainer" containerID="bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b" Jul 08 16:20:13.777294 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.777264 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:20:13.777403 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.777349 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:20:13.777403 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:13.777394 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:20:14.782861 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:14.782826 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerStarted","Data":"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a"} Jul 08 16:20:14.910765 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:14.910731 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:14.912190 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:14.912137 2566 patch_prober.go:28] interesting pod/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 16:20:14.912340 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:14.912217 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 16:20:15.683928 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:15.683893 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:15.911113 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:15.911067 2566 patch_prober.go:28] interesting pod/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 16:20:15.911724 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:15.911144 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 16:20:16.911428 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:16.911384 2566 patch_prober.go:28] interesting pod/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 16:20:16.911942 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:16.911470 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 16:20:17.910984 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:17.910946 2566 patch_prober.go:28] interesting pod/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 16:20:17.911180 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:17.911030 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 16:20:18.914371 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:18.914341 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:18.915483 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:18.915460 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:20:48.188866 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.188779 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:48.189407 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.189025 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" podUID="922a760a-aa35-4ead-8104-7bf9b88bad37" containerName="maas-api" containerID="cri-o://0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063" gracePeriod=30 Jul 08 16:20:48.435899 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.435876 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:48.509402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.509332 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls\") pod \"922a760a-aa35-4ead-8104-7bf9b88bad37\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " Jul 08 16:20:48.509402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.509363 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca\") pod \"922a760a-aa35-4ead-8104-7bf9b88bad37\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " Jul 08 16:20:48.509402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.509401 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4rdw\" (UniqueName: \"kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw\") pod \"922a760a-aa35-4ead-8104-7bf9b88bad37\" (UID: \"922a760a-aa35-4ead-8104-7bf9b88bad37\") " Jul 08 16:20:48.509795 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.509766 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "922a760a-aa35-4ead-8104-7bf9b88bad37" (UID: "922a760a-aa35-4ead-8104-7bf9b88bad37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:20:48.511652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.511623 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "922a760a-aa35-4ead-8104-7bf9b88bad37" (UID: "922a760a-aa35-4ead-8104-7bf9b88bad37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:20:48.511652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.511636 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw" (OuterVolumeSpecName: "kube-api-access-b4rdw") pod "922a760a-aa35-4ead-8104-7bf9b88bad37" (UID: "922a760a-aa35-4ead-8104-7bf9b88bad37"). InnerVolumeSpecName "kube-api-access-b4rdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:20:48.610388 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.610363 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4rdw\" (UniqueName: \"kubernetes.io/projected/922a760a-aa35-4ead-8104-7bf9b88bad37-kube-api-access-b4rdw\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:20:48.610388 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.610387 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922a760a-aa35-4ead-8104-7bf9b88bad37-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:20:48.610524 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.610396 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922a760a-aa35-4ead-8104-7bf9b88bad37-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:20:48.953490 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.953462 2566 generic.go:356] "Generic (PLEG): container finished" podID="922a760a-aa35-4ead-8104-7bf9b88bad37" containerID="0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063" exitCode=0 Jul 08 16:20:48.953652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.953510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" event={"ID":"922a760a-aa35-4ead-8104-7bf9b88bad37","Type":"ContainerDied","Data":"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063"} Jul 08 16:20:48.953652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.953512 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" Jul 08 16:20:48.953652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.953615 2566 scope.go:117] "RemoveContainer" containerID="0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063" Jul 08 16:20:48.953757 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.953603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss" event={"ID":"922a760a-aa35-4ead-8104-7bf9b88bad37","Type":"ContainerDied","Data":"f7bd1f3d7482a66a1fa67d9ee3612a29d9a22a097e44e1283187f07e3eab3dd2"} Jul 08 16:20:48.963402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.963384 2566 scope.go:117] "RemoveContainer" containerID="0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063" Jul 08 16:20:48.963682 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:20:48.963665 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063\": container with ID starting with 0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063 not found: ID does not exist" containerID="0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063" Jul 08 16:20:48.963739 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.963690 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063"} err="failed to get container status \"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063\": rpc error: code = NotFound desc = could not find container \"0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063\": container with ID starting with 0bd01c45275a3bb3420a54717917ef834493fce36bb1bc12b775786fd0618063 not found: ID does not exist" Jul 08 16:20:48.975667 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.975649 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:48.978063 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:48.978047 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68603676-56cff6d9dd-v8bss"] Jul 08 16:20:49.268805 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:49.268718 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922a760a-aa35-4ead-8104-7bf9b88bad37" path="/var/lib/kubelet/pods/922a760a-aa35-4ead-8104-7bf9b88bad37/volumes" Jul 08 16:20:55.295056 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:55.295029 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:20:55.300664 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:55.300641 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:20:55.302098 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:55.302076 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:20:55.307189 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:20:55.307170 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:21:47.875703 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:47.875656 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:21:47.876238 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:47.875912 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" containerID="cri-o://90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a" gracePeriod=30 Jul 08 16:21:48.916851 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:48.916816 2566 patch_prober.go:28] interesting pod/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 16:21:48.917222 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:48.916873 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 16:21:53.116947 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.116923 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:21:53.258022 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.257937 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258030 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258049 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258066 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258083 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258124 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258147 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258494 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258176 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7hkd\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258494 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258203 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token\") pod \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\" (UID: \"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25\") " Jul 08 16:21:53.258494 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258197 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:21:53.258494 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258410 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:21:53.258494 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258441 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:21:53.258798 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.258557 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:21:53.259377 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.259346 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data" (OuterVolumeSpecName: "istio-data") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:21:53.260434 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.260406 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:21:53.260525 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.260434 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:21:53.260628 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.260555 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd" (OuterVolumeSpecName: "kube-api-access-n7hkd") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "kube-api-access-n7hkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:21:53.260678 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.260623 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token" (OuterVolumeSpecName: "istio-token") pod "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" (UID: "589f2fad-e5e9-4d92-b5d3-65da9ebb1c25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:21:53.278587 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.278537 2566 generic.go:356] "Generic (PLEG): container finished" podID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerID="90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a" exitCode=0 Jul 08 16:21:53.278690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.278590 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" Jul 08 16:21:53.278690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.278614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerDied","Data":"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a"} Jul 08 16:21:53.278690 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.278676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm" event={"ID":"589f2fad-e5e9-4d92-b5d3-65da9ebb1c25","Type":"ContainerDied","Data":"a76891ceb80c38e4ff81907149fd9c27d6df3c35c015aad918572edb23ce8ef4"} Jul 08 16:21:53.278857 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.278708 2566 scope.go:117] "RemoveContainer" containerID="90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a" Jul 08 16:21:53.287486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.287470 2566 scope.go:117] "RemoveContainer" containerID="bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b" Jul 08 16:21:53.297503 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.297479 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:21:53.298769 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.298748 2566 scope.go:117] "RemoveContainer" containerID="90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a" Jul 08 16:21:53.299036 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:21:53.299018 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a\": container with ID starting with 90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a not found: ID does not exist" containerID="90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a" Jul 08 16:21:53.299092 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.299044 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a"} err="failed to get container status \"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a\": rpc error: code = NotFound desc = could not find container \"90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a\": container with ID starting with 90caeceb1a820cb4940053f953123c4353d24f527e0d1e951a04134c9cbc488a not found: ID does not exist" Jul 08 16:21:53.299092 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.299061 2566 scope.go:117] "RemoveContainer" containerID="bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b" Jul 08 16:21:53.299282 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:21:53.299262 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b\": container with ID starting with bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b not found: ID does not exist" containerID="bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b" Jul 08 16:21:53.299337 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.299290 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b"} err="failed to get container status \"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b\": rpc error: code = NotFound desc = could not find container \"bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b\": container with ID starting with bf92b0ffb7030c3a5894fe3c2e39e8ecf9377259b2ccb31a209d5ed788cecf7b not found: ID does not exist" Jul 08 16:21:53.303509 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.303488 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-68603676-openshift-default-5f6cd85ff-s9vnm"] Jul 08 16:21:53.359057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359033 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359052 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359057 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359060 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359069 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359077 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359088 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359097 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7hkd\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-kube-api-access-n7hkd\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359105 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:53.359197 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:53.359112 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:21:55.268634 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:21:55.268599 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" path="/var/lib/kubelet/pods/589f2fad-e5e9-4d92-b5d3-65da9ebb1c25/volumes" Jul 08 16:22:43.603822 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.603792 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604100 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604111 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604125 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922a760a-aa35-4ead-8104-7bf9b88bad37" containerName="maas-api" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604131 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="922a760a-aa35-4ead-8104-7bf9b88bad37" containerName="maas-api" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604139 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604145 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604199 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604206 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="589f2fad-e5e9-4d92-b5d3-65da9ebb1c25" containerName="istio-proxy" Jul 08 16:22:43.604257 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.604213 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="922a760a-aa35-4ead-8104-7bf9b88bad37" containerName="maas-api" Jul 08 16:22:43.607104 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.607087 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.609380 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.609362 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2139f3-openshift-default-dockercfg-sq9fd\"" Jul 08 16:22:43.621496 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.621473 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:22:43.744330 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744330 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744496 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w7wq\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744764 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744764 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.744764 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.744617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845187 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845202 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w7wq\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845358 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845600 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845657 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845727 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845972 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845886 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.845972 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.845938 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.847722 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.847702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.847969 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.847951 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.852999 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.852973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w7wq\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.853492 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.853476 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token\") pod \"e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:43.916822 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:43.916767 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:44.052069 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.052038 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:22:44.053546 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:22:44.053519 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31edca59_dc43_4a2f_9d92_ee6804964c53.slice/crio-84025f7655e7467fc65d549170f3d1724cc9df4a9b963480c28096b5b80d138f WatchSource:0}: Error finding container 84025f7655e7467fc65d549170f3d1724cc9df4a9b963480c28096b5b80d138f: Status 404 returned error can't find the container with id 84025f7655e7467fc65d549170f3d1724cc9df4a9b963480c28096b5b80d138f Jul 08 16:22:44.055782 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.055748 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:22:44.055856 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.055813 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:22:44.055856 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.055844 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:22:44.540010 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.539968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerStarted","Data":"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb"} Jul 08 16:22:44.540211 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.540032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerStarted","Data":"84025f7655e7467fc65d549170f3d1724cc9df4a9b963480c28096b5b80d138f"} Jul 08 16:22:44.561215 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.561170 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podStartSLOduration=1.5611559879999999 podStartE2EDuration="1.561155988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:22:44.558844462 +0000 UTC m=+1909.821863131" watchObservedRunningTime="2026-07-08 16:22:44.561155988 +0000 UTC m=+1909.824174658" Jul 08 16:22:44.917426 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:44.917395 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:45.921756 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:45.921728 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:46.550546 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.550512 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:46.551506 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.551482 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:22:46.726254 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.726225 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:22:46.729780 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.729761 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.732109 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.732087 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:22:46.732212 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.732129 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2139f3-serving-cert\"" Jul 08 16:22:46.732457 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.732359 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:22:46.732457 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.732365 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:22:46.743625 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.743607 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:22:46.874217 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.874187 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.874374 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.874249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.874374 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.874265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg8r4\" (UniqueName: \"kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.974866 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.974836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.975218 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.974897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.975218 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.974916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg8r4\" (UniqueName: \"kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.975218 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:46.975054 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-serving-cert: secret "maas-api-e2e-inf-a-2139f3-serving-cert" not found Jul 08 16:22:46.975218 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:46.975119 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls podName:e0187510-24dc-4a4d-871e-e89c821d7df7 nodeName:}" failed. No retries permitted until 2026-07-08 16:22:47.475097916 +0000 UTC m=+1912.738116564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls") pod "maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" (UID: "e0187510-24dc-4a4d-871e-e89c821d7df7") : secret "maas-api-e2e-inf-a-2139f3-serving-cert" not found Jul 08 16:22:46.975558 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.975538 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:46.983462 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:46.983444 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg8r4\" (UniqueName: \"kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:47.479878 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:47.479846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:47.480068 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:47.479972 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-serving-cert: secret "maas-api-e2e-inf-a-2139f3-serving-cert" not found Jul 08 16:22:47.480068 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:47.480034 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls podName:e0187510-24dc-4a4d-871e-e89c821d7df7 nodeName:}" failed. No retries permitted until 2026-07-08 16:22:48.480019505 +0000 UTC m=+1913.743038156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls") pod "maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" (UID: "e0187510-24dc-4a4d-871e-e89c821d7df7") : secret "maas-api-e2e-inf-a-2139f3-serving-cert" not found Jul 08 16:22:48.487337 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:48.487301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:48.489938 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:48.489913 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:48.541831 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:48.541802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:48.682170 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:48.682146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:22:48.683968 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:22:48.683943 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0187510_24dc_4a4d_871e_e89c821d7df7.slice/crio-469080ece85ab9fd7bc91eb576bc8eba50aa71fa0f42f5cd29c3b1ec1fae4ef5 WatchSource:0}: Error finding container 469080ece85ab9fd7bc91eb576bc8eba50aa71fa0f42f5cd29c3b1ec1fae4ef5: Status 404 returned error can't find the container with id 469080ece85ab9fd7bc91eb576bc8eba50aa71fa0f42f5cd29c3b1ec1fae4ef5 Jul 08 16:22:48.685899 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:48.685881 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:22:49.566867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:49.566776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" event={"ID":"e0187510-24dc-4a4d-871e-e89c821d7df7","Type":"ContainerStarted","Data":"b1ad59f3b1118026d37c9968913e9d0a3e661246f4f53ca432df9a1989a0bd9b"} Jul 08 16:22:49.566867 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:49.566829 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" event={"ID":"e0187510-24dc-4a4d-871e-e89c821d7df7","Type":"ContainerStarted","Data":"469080ece85ab9fd7bc91eb576bc8eba50aa71fa0f42f5cd29c3b1ec1fae4ef5"} Jul 08 16:22:49.567284 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:49.566904 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:49.594607 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:49.594540 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" podStartSLOduration=3.14717581 podStartE2EDuration="3.594522608s" totalImagesPullingTime="447.346798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:22:49.592793495 +0000 UTC m=+1914.855812175" watchObservedRunningTime="2026-07-08 16:22:49.594522608 +0000 UTC m=+1914.857541280" Jul 08 16:22:55.172994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.172957 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:22:55.176634 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.176609 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.179892 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.179869 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-029815-serving-cert\"" Jul 08 16:22:55.192214 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.192190 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:22:55.347432 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.347400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.347625 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.347442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmbvr\" (UniqueName: \"kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.347625 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.347474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.448349 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.448268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.448493 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.448345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.448493 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.448380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbvr\" (UniqueName: \"kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.449149 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.449102 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.450831 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.450808 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.456899 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.456866 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmbvr\" (UniqueName: \"kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr\") pod \"maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.487832 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.487802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:22:55.579263 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.579236 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:22:55.626076 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:55.626051 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:22:55.628362 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:22:55.628334 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa8d69e_1bbe_4403_aaa1_526b80f7f556.slice/crio-d225a2a60041c148a768d8787ef642d311e56a4b016d73b719faa86e2afa0ee8 WatchSource:0}: Error finding container d225a2a60041c148a768d8787ef642d311e56a4b016d73b719faa86e2afa0ee8: Status 404 returned error can't find the container with id d225a2a60041c148a768d8787ef642d311e56a4b016d73b719faa86e2afa0ee8 Jul 08 16:22:56.129532 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:56.129477 2566 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/opendatahub/maas-api@sha256:7c71a85355ca9f3de4d31fb3b8c6b042f704d773e07580693a1007c94b04623e" Jul 08 16:22:56.129804 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:56.129662 2566 kuberuntime_manager.go:1459] "Unhandled Error" err="container maas-api start failed in pod maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr_odh-ai-gateway-infra(9fa8d69e-1bbe-4403-aaa1-526b80f7f556): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 08 16:22:56.129804 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:56.129705 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" Jul 08 16:22:56.603591 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:22:56.603532 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" event={"ID":"9fa8d69e-1bbe-4403-aaa1-526b80f7f556","Type":"ContainerStarted","Data":"d225a2a60041c148a768d8787ef642d311e56a4b016d73b719faa86e2afa0ee8"} Jul 08 16:22:56.604098 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:56.604072 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/maas-api@sha256:7c71a85355ca9f3de4d31fb3b8c6b042f704d773e07580693a1007c94b04623e\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" Jul 08 16:22:57.609161 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:22:57.609125 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/maas-api@sha256:7c71a85355ca9f3de4d31fb3b8c6b042f704d773e07580693a1007c94b04623e\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" Jul 08 16:23:02.592790 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:02.592688 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 16:23:02.596414 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:02.592835 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 16:23:03.651372 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.651341 2566 generic.go:356] "Generic (PLEG): container finished" podID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerID="9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb" exitCode=0 Jul 08 16:23:03.651872 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.651424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerDied","Data":"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb"} Jul 08 16:23:03.651872 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.651827 2566 scope.go:117] "RemoveContainer" containerID="9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb" Jul 08 16:23:03.652869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.652841 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:23:03.652940 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.652922 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:23:03.652990 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:03.652975 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:23:04.659994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:04.659961 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerStarted","Data":"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f"} Jul 08 16:23:04.917794 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:04.917692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:23:04.918141 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:04.918113 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 16:23:04.918228 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:04.918187 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 16:23:05.917947 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:05.917912 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 16:23:05.918541 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:05.918506 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 16:23:06.917534 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:06.917497 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 16:23:06.917837 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:06.917808 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 16:23:07.918516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:07.918062 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 16:23:07.918516 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:07.918144 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 16:23:08.921068 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:08.921036 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:23:08.921519 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:08.921318 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:23:08.922061 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:08.922044 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:23:12.703421 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:12.703392 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" event={"ID":"9fa8d69e-1bbe-4403-aaa1-526b80f7f556","Type":"ContainerStarted","Data":"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f"} Jul 08 16:23:12.703717 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:12.703610 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:23:12.724634 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:12.724584 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" podStartSLOduration=0.737145245 podStartE2EDuration="17.724554996s" totalImagesPullingTime="16.987409751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 16:23:12.722631491 +0000 UTC m=+1937.985650153" watchObservedRunningTime="2026-07-08 16:23:12.724554996 +0000 UTC m=+1937.987573664" Jul 08 16:23:18.713168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:23:18.713135 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:24:05.776749 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:05.776508 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:24:05.777326 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:05.776834 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" podUID="e0187510-24dc-4a4d-871e-e89c821d7df7" containerName="maas-api" containerID="cri-o://b1ad59f3b1118026d37c9968913e9d0a3e661246f4f53ca432df9a1989a0bd9b" gracePeriod=30 Jul 08 16:24:05.976753 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:05.976722 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0187510-24dc-4a4d-871e-e89c821d7df7" containerID="b1ad59f3b1118026d37c9968913e9d0a3e661246f4f53ca432df9a1989a0bd9b" exitCode=0 Jul 08 16:24:05.976922 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:05.976796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" event={"ID":"e0187510-24dc-4a4d-871e-e89c821d7df7","Type":"ContainerDied","Data":"b1ad59f3b1118026d37c9968913e9d0a3e661246f4f53ca432df9a1989a0bd9b"} Jul 08 16:24:06.021927 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.021897 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:24:06.134377 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.134353 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca\") pod \"e0187510-24dc-4a4d-871e-e89c821d7df7\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " Jul 08 16:24:06.134529 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.134382 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") pod \"e0187510-24dc-4a4d-871e-e89c821d7df7\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " Jul 08 16:24:06.134529 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.134452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg8r4\" (UniqueName: \"kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4\") pod \"e0187510-24dc-4a4d-871e-e89c821d7df7\" (UID: \"e0187510-24dc-4a4d-871e-e89c821d7df7\") " Jul 08 16:24:06.134743 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.134721 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0187510-24dc-4a4d-871e-e89c821d7df7" (UID: "e0187510-24dc-4a4d-871e-e89c821d7df7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:24:06.136693 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.136672 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0187510-24dc-4a4d-871e-e89c821d7df7" (UID: "e0187510-24dc-4a4d-871e-e89c821d7df7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:24:06.136769 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.136693 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4" (OuterVolumeSpecName: "kube-api-access-qg8r4") pod "e0187510-24dc-4a4d-871e-e89c821d7df7" (UID: "e0187510-24dc-4a4d-871e-e89c821d7df7"). InnerVolumeSpecName "kube-api-access-qg8r4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:24:06.235869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.235840 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0187510-24dc-4a4d-871e-e89c821d7df7-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:06.235869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.235867 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0187510-24dc-4a4d-871e-e89c821d7df7-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:06.236031 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.235876 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg8r4\" (UniqueName: \"kubernetes.io/projected/e0187510-24dc-4a4d-871e-e89c821d7df7-kube-api-access-qg8r4\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:06.982965 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.982929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" event={"ID":"e0187510-24dc-4a4d-871e-e89c821d7df7","Type":"ContainerDied","Data":"469080ece85ab9fd7bc91eb576bc8eba50aa71fa0f42f5cd29c3b1ec1fae4ef5"} Jul 08 16:24:06.982965 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.982957 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn" Jul 08 16:24:06.983439 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:06.983000 2566 scope.go:117] "RemoveContainer" containerID="b1ad59f3b1118026d37c9968913e9d0a3e661246f4f53ca432df9a1989a0bd9b" Jul 08 16:24:07.004450 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:07.004428 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:24:07.008311 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:07.008288 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2139f3-84c4c69f46-7z8rn"] Jul 08 16:24:07.274864 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:07.274779 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0187510-24dc-4a4d-871e-e89c821d7df7" path="/var/lib/kubelet/pods/e0187510-24dc-4a4d-871e-e89c821d7df7/volumes" Jul 08 16:24:48.155014 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:48.154976 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:24:48.155482 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:48.155218 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" containerID="cri-o://4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f" gracePeriod=30 Jul 08 16:24:49.466946 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.466909 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:24:49.467314 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.467113 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" containerName="maas-api" containerID="cri-o://a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f" gracePeriod=30 Jul 08 16:24:49.726252 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.726188 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:24:49.881376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.881346 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmbvr\" (UniqueName: \"kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr\") pod \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " Jul 08 16:24:49.881541 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.881399 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls\") pod \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " Jul 08 16:24:49.881541 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.881423 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca\") pod \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\" (UID: \"9fa8d69e-1bbe-4403-aaa1-526b80f7f556\") " Jul 08 16:24:49.881865 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.881830 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9fa8d69e-1bbe-4403-aaa1-526b80f7f556" (UID: "9fa8d69e-1bbe-4403-aaa1-526b80f7f556"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:24:49.883533 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.883509 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9fa8d69e-1bbe-4403-aaa1-526b80f7f556" (UID: "9fa8d69e-1bbe-4403-aaa1-526b80f7f556"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:24:49.883646 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.883559 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr" (OuterVolumeSpecName: "kube-api-access-fmbvr") pod "9fa8d69e-1bbe-4403-aaa1-526b80f7f556" (UID: "9fa8d69e-1bbe-4403-aaa1-526b80f7f556"). InnerVolumeSpecName "kube-api-access-fmbvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:24:49.982087 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.982017 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmbvr\" (UniqueName: \"kubernetes.io/projected/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-kube-api-access-fmbvr\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:49.982087 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.982047 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:49.982087 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:49.982060 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8d69e-1bbe-4403-aaa1-526b80f7f556-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:50.210127 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.210092 2566 generic.go:356] "Generic (PLEG): container finished" podID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" containerID="a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f" exitCode=0 Jul 08 16:24:50.210290 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.210138 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" Jul 08 16:24:50.210290 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.210181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" event={"ID":"9fa8d69e-1bbe-4403-aaa1-526b80f7f556","Type":"ContainerDied","Data":"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f"} Jul 08 16:24:50.210290 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.210265 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr" event={"ID":"9fa8d69e-1bbe-4403-aaa1-526b80f7f556","Type":"ContainerDied","Data":"d225a2a60041c148a768d8787ef642d311e56a4b016d73b719faa86e2afa0ee8"} Jul 08 16:24:50.210441 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.210294 2566 scope.go:117] "RemoveContainer" containerID="a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f" Jul 08 16:24:50.220042 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.220026 2566 scope.go:117] "RemoveContainer" containerID="a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f" Jul 08 16:24:50.220298 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:24:50.220280 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f\": container with ID starting with a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f not found: ID does not exist" containerID="a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f" Jul 08 16:24:50.220347 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.220307 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f"} err="failed to get container status \"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f\": rpc error: code = NotFound desc = could not find container \"a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f\": container with ID starting with a29f67ca05f9fad8ef0a4c63161731fdf953691449bd36459f3abe20e8c98d4f not found: ID does not exist" Jul 08 16:24:50.233362 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.233313 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:24:50.237549 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:50.237529 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-029815-6d846f9c6b-5qpdr"] Jul 08 16:24:51.269745 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:51.269714 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" path="/var/lib/kubelet/pods/9fa8d69e-1bbe-4403-aaa1-526b80f7f556/volumes" Jul 08 16:24:53.417241 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.417182 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:24:53.508820 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.508783 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.508998 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.508860 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.508998 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.508881 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509061 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w7wq\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509173 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509108 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509173 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509154 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509173 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509157 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:24:53.509323 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509186 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509323 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509244 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509323 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509283 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token\") pod \"31edca59-dc43-4a2f-9d92-ee6804964c53\" (UID: \"31edca59-dc43-4a2f-9d92-ee6804964c53\") " Jul 08 16:24:53.509459 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509244 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:24:53.509511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509441 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:24:53.509511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509498 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:24:53.509644 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509625 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.509687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509651 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31edca59-dc43-4a2f-9d92-ee6804964c53-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.509687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509666 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.509687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509679 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.509986 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.509964 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data" (OuterVolumeSpecName: "istio-data") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:24:53.511346 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.511321 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:24:53.511451 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.511373 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq" (OuterVolumeSpecName: "kube-api-access-5w7wq") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "kube-api-access-5w7wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:24:53.511618 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.511598 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token" (OuterVolumeSpecName: "istio-token") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:24:53.511952 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.511939 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31edca59-dc43-4a2f-9d92-ee6804964c53" (UID: "31edca59-dc43-4a2f-9d92-ee6804964c53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:24:53.610542 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.610506 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.610542 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.610535 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.610542 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.610545 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w7wq\" (UniqueName: \"kubernetes.io/projected/31edca59-dc43-4a2f-9d92-ee6804964c53-kube-api-access-5w7wq\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.610797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.610553 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:53.610797 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:53.610562 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31edca59-dc43-4a2f-9d92-ee6804964c53-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:24:54.235637 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.235605 2566 generic.go:356] "Generic (PLEG): container finished" podID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerID="4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f" exitCode=0 Jul 08 16:24:54.235841 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.235651 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" Jul 08 16:24:54.235841 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.235689 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerDied","Data":"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f"} Jul 08 16:24:54.235841 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.235756 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8" event={"ID":"31edca59-dc43-4a2f-9d92-ee6804964c53","Type":"ContainerDied","Data":"84025f7655e7467fc65d549170f3d1724cc9df4a9b963480c28096b5b80d138f"} Jul 08 16:24:54.235841 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.235785 2566 scope.go:117] "RemoveContainer" containerID="4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f" Jul 08 16:24:54.245217 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.245200 2566 scope.go:117] "RemoveContainer" containerID="9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb" Jul 08 16:24:54.253073 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.253056 2566 scope.go:117] "RemoveContainer" containerID="4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f" Jul 08 16:24:54.253311 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:24:54.253291 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f\": container with ID starting with 4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f not found: ID does not exist" containerID="4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f" Jul 08 16:24:54.253402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.253319 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f"} err="failed to get container status \"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f\": rpc error: code = NotFound desc = could not find container \"4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f\": container with ID starting with 4ae200ac193bac68a66272b7cd852f97e90f3cde2faa8cfc96a109a505ab883f not found: ID does not exist" Jul 08 16:24:54.253402 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.253334 2566 scope.go:117] "RemoveContainer" containerID="9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb" Jul 08 16:24:54.253609 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:24:54.253591 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb\": container with ID starting with 9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb not found: ID does not exist" containerID="9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb" Jul 08 16:24:54.253664 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.253617 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb"} err="failed to get container status \"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb\": rpc error: code = NotFound desc = could not find container \"9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb\": container with ID starting with 9c0eafedf754786dd62c60a76233a8ad885a10b7afa6a911a6e41f6b7bc5eebb not found: ID does not exist" Jul 08 16:24:54.257038 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.257018 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:24:54.264560 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:54.264534 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2139f3-openshift-default-6466b84bd5-6jwf8"] Jul 08 16:24:55.268461 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:55.268427 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" path="/var/lib/kubelet/pods/31edca59-dc43-4a2f-9d92-ee6804964c53/volumes" Jul 08 16:24:56.323241 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323213 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323539 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323549 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323562 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0187510-24dc-4a4d-871e-e89c821d7df7" containerName="maas-api" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323586 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0187510-24dc-4a4d-871e-e89c821d7df7" containerName="maas-api" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323599 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323605 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323618 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" containerName="maas-api" Jul 08 16:24:56.323624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323624 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" containerName="maas-api" Jul 08 16:24:56.323869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323670 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0187510-24dc-4a4d-871e-e89c821d7df7" containerName="maas-api" Jul 08 16:24:56.323869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323680 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.323869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323685 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa8d69e-1bbe-4403-aaa1-526b80f7f556" containerName="maas-api" Jul 08 16:24:56.323869 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.323691 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="31edca59-dc43-4a2f-9d92-ee6804964c53" containerName="istio-proxy" Jul 08 16:24:56.328248 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.328231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.330555 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.330538 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a3f98e-openshift-default-dockercfg-fm5vm\"" Jul 08 16:24:56.339167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.339142 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:24:56.432417 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432586 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432586 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9v6v\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432586 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432586 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432768 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432768 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.432852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.432834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534270 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534416 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534416 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534416 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534547 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534547 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534547 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9v6v\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534725 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534725 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534725 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534629 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534725 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534681 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.534934 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.534794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.535143 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.535114 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.535254 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.535229 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.537029 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.537012 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.537163 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.537144 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.541733 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.541714 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.542051 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.542029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9v6v\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v\") pod \"e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.639773 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.639746 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:56.776939 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.776909 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:24:56.778863 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:24:56.778836 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f7a7e3_55ce_42af_91d3_d419663c8cfe.slice/crio-32f41c4ef0702dfb9155e82fdacd1c37ca521789dfb3c895822ffaf79464a6eb WatchSource:0}: Error finding container 32f41c4ef0702dfb9155e82fdacd1c37ca521789dfb3c895822ffaf79464a6eb: Status 404 returned error can't find the container with id 32f41c4ef0702dfb9155e82fdacd1c37ca521789dfb3c895822ffaf79464a6eb Jul 08 16:24:56.781183 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.781151 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:24:56.781278 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.781213 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:24:56.781278 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:56.781240 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:24:57.253963 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:57.253929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerStarted","Data":"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a"} Jul 08 16:24:57.254112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:57.253986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerStarted","Data":"32f41c4ef0702dfb9155e82fdacd1c37ca521789dfb3c895822ffaf79464a6eb"} Jul 08 16:24:57.276945 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:57.276901 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podStartSLOduration=1.276885303 podStartE2EDuration="1.276885303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:24:57.274873542 +0000 UTC m=+2042.537892211" watchObservedRunningTime="2026-07-08 16:24:57.276885303 +0000 UTC m=+2042.539903972" Jul 08 16:24:57.640190 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:57.640159 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:57.645120 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:57.645096 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:58.258941 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:58.258911 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:58.259740 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:58.259720 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:24:59.324859 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.324827 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:24:59.328043 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.328025 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.336823 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.336799 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:24:59.337940 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.337921 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:24:59.338247 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.338230 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:24:59.338633 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.338616 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a3f98e-serving-cert\"" Jul 08 16:24:59.340193 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.340170 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:24:59.459130 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.458976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.459130 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.459023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.459403 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.459138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shksn\" (UniqueName: \"kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.560396 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.560366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shksn\" (UniqueName: \"kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.560558 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.560418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.560558 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.560437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.561129 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.561101 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.563016 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.562990 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.570334 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.570309 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shksn\" (UniqueName: \"kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn\") pod \"maas-api-e2e-api-a-a3f98e-565686d5c-82l9m\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.645857 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.645833 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:24:59.778922 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:24:59.778898 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:24:59.779970 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:24:59.779930 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f91fa5_19d3_4cb0_bdd2_c7fd8244c290.slice/crio-0315f27aafd0cd905834073e488376dc0a547ebba6d61aac62badb6d2cbdc505 WatchSource:0}: Error finding container 0315f27aafd0cd905834073e488376dc0a547ebba6d61aac62badb6d2cbdc505: Status 404 returned error can't find the container with id 0315f27aafd0cd905834073e488376dc0a547ebba6d61aac62badb6d2cbdc505 Jul 08 16:25:00.269851 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:00.269824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" event={"ID":"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290","Type":"ContainerStarted","Data":"0315f27aafd0cd905834073e488376dc0a547ebba6d61aac62badb6d2cbdc505"} Jul 08 16:25:01.275890 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:01.275851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" event={"ID":"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290","Type":"ContainerStarted","Data":"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189"} Jul 08 16:25:01.276287 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:01.275919 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:25:01.297066 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:01.297020 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" podStartSLOduration=1.830570509 podStartE2EDuration="2.297005222s" totalImagesPullingTime="466.434713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:25:01.295156196 +0000 UTC m=+2046.558174876" watchObservedRunningTime="2026-07-08 16:25:01.297005222 +0000 UTC m=+2046.560023890" Jul 08 16:25:07.286726 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:07.286699 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:25:09.140220 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.140170 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:09.144288 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.144266 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.150625 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.150603 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5689eb-serving-cert\"" Jul 08 16:25:09.184362 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.184327 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:09.340799 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.340757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4v9\" (UniqueName: \"kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.340799 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.340801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.341010 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.340820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.442433 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.442346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4v9\" (UniqueName: \"kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.442433 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.442387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.442433 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.442409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.443162 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.443128 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.444994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.444975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.455117 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.455091 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4v9\" (UniqueName: \"kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9\") pod \"maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.456984 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.456964 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:09.637475 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:09.637387 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:09.641163 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:25:09.641127 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7c2804_ea72_4f01_9d48_cdc9a6f7c969.slice/crio-4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c WatchSource:0}: Error finding container 4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c: Status 404 returned error can't find the container with id 4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c Jul 08 16:25:10.335636 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:10.335508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" event={"ID":"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969","Type":"ContainerStarted","Data":"2fd9caa61ca2b93630c73d469f5e0f1d367a95038ffb0ebe47f41940ea7a7ce1"} Jul 08 16:25:10.335636 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:10.335607 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" event={"ID":"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969","Type":"ContainerStarted","Data":"4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c"} Jul 08 16:25:10.336170 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:10.335773 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:10.355561 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:10.355517 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" podStartSLOduration=1.017816553 podStartE2EDuration="1.355502876s" totalImagesPullingTime="337.686323ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:25:10.353086305 +0000 UTC m=+2055.616104974" watchObservedRunningTime="2026-07-08 16:25:10.355502876 +0000 UTC m=+2055.618521544" Jul 08 16:25:14.300787 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:14.300736 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 16:25:14.301187 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:14.300806 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 16:25:16.348206 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.348123 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:16.371549 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.371512 2566 generic.go:356] "Generic (PLEG): container finished" podID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerID="8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a" exitCode=0 Jul 08 16:25:16.371728 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.371596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerDied","Data":"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a"} Jul 08 16:25:16.372029 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.372013 2566 scope.go:117] "RemoveContainer" containerID="8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a" Jul 08 16:25:16.373139 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.373108 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:16.373248 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.373172 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:16.373248 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:16.373211 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:17.378366 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:17.378322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerStarted","Data":"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012"} Jul 08 16:25:17.639948 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:17.639865 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:17.640094 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:17.640063 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 16:25:17.640141 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:17.640103 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 16:25:18.640622 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:18.640553 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 16:25:18.641042 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:18.640660 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 16:25:19.640322 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:19.640287 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 16:25:19.640598 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:19.640341 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 16:25:20.640182 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:20.640147 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 16:25:20.640594 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:20.640201 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 16:25:21.640242 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:21.640202 2566 patch_prober.go:28] interesting pod/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 16:25:21.640654 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:21.640261 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 16:25:22.011169 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.011082 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:25:22.011323 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.011301 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" podUID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" containerName="maas-api" containerID="cri-o://e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189" gracePeriod=30 Jul 08 16:25:22.256740 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.256718 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:25:22.357966 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.357939 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shksn\" (UniqueName: \"kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn\") pod \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " Jul 08 16:25:22.358098 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.357973 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls\") pod \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " Jul 08 16:25:22.358098 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.358027 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca\") pod \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\" (UID: \"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290\") " Jul 08 16:25:22.358429 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.358402 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" (UID: "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:25:22.360198 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.360174 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" (UID: "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:25:22.360279 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.360203 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn" (OuterVolumeSpecName: "kube-api-access-shksn") pod "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" (UID: "02f91fa5-19d3-4cb0-bdd2-c7fd8244c290"). InnerVolumeSpecName "kube-api-access-shksn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:25:22.406011 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.405985 2566 generic.go:356] "Generic (PLEG): container finished" podID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" containerID="e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189" exitCode=0 Jul 08 16:25:22.406118 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.406026 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" Jul 08 16:25:22.406118 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.406080 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" event={"ID":"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290","Type":"ContainerDied","Data":"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189"} Jul 08 16:25:22.406239 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.406141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m" event={"ID":"02f91fa5-19d3-4cb0-bdd2-c7fd8244c290","Type":"ContainerDied","Data":"0315f27aafd0cd905834073e488376dc0a547ebba6d61aac62badb6d2cbdc505"} Jul 08 16:25:22.406239 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.406174 2566 scope.go:117] "RemoveContainer" containerID="e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189" Jul 08 16:25:22.414962 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.414945 2566 scope.go:117] "RemoveContainer" containerID="e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189" Jul 08 16:25:22.415200 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:25:22.415182 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189\": container with ID starting with e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189 not found: ID does not exist" containerID="e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189" Jul 08 16:25:22.415249 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.415209 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189"} err="failed to get container status \"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189\": rpc error: code = NotFound desc = could not find container \"e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189\": container with ID starting with e80cfd339c6e3bfd2e03e93d7218fd7f429b3cead28e5a6cf7ca34b6400e7189 not found: ID does not exist" Jul 08 16:25:22.431536 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.431513 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:25:22.437613 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.437594 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a3f98e-565686d5c-82l9m"] Jul 08 16:25:22.459139 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.459118 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shksn\" (UniqueName: \"kubernetes.io/projected/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-kube-api-access-shksn\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:22.459139 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.459135 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:22.459256 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.459145 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:22.644290 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.644206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:22.644741 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.644661 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:22.645610 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:22.645563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:23.269116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:23.269080 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" path="/var/lib/kubelet/pods/02f91fa5-19d3-4cb0-bdd2-c7fd8244c290/volumes" Jul 08 16:25:29.033895 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:29.033859 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:25:29.034315 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:29.034066 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" containerID="cri-o://10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012" gracePeriod=30 Jul 08 16:25:30.218082 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.218043 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:30.218461 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.218301 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" podUID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" containerName="maas-api" containerID="cri-o://2fd9caa61ca2b93630c73d469f5e0f1d367a95038ffb0ebe47f41940ea7a7ce1" gracePeriod=30 Jul 08 16:25:30.447900 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.447875 2566 generic.go:356] "Generic (PLEG): container finished" podID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" containerID="2fd9caa61ca2b93630c73d469f5e0f1d367a95038ffb0ebe47f41940ea7a7ce1" exitCode=0 Jul 08 16:25:30.448020 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.447950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" event={"ID":"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969","Type":"ContainerDied","Data":"2fd9caa61ca2b93630c73d469f5e0f1d367a95038ffb0ebe47f41940ea7a7ce1"} Jul 08 16:25:30.448020 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.448013 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" event={"ID":"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969","Type":"ContainerDied","Data":"4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c"} Jul 08 16:25:30.448107 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.448035 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4843a0669d0e6cb04a0355d8a9d01568a2a53a4f06e23ee7f36015b8fe6a398c" Jul 08 16:25:30.458164 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.458147 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:30.526149 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.526081 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca\") pod \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " Jul 08 16:25:30.526149 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.526137 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz4v9\" (UniqueName: \"kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9\") pod \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " Jul 08 16:25:30.526300 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.526164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls\") pod \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\" (UID: \"8f7c2804-ea72-4f01-9d48-cdc9a6f7c969\") " Jul 08 16:25:30.526484 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.526451 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" (UID: "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:25:30.528264 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.528247 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" (UID: "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:25:30.528409 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.528391 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9" (OuterVolumeSpecName: "kube-api-access-nz4v9") pod "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" (UID: "8f7c2804-ea72-4f01-9d48-cdc9a6f7c969"). InnerVolumeSpecName "kube-api-access-nz4v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:25:30.627116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.627091 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:30.627116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.627111 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz4v9\" (UniqueName: \"kubernetes.io/projected/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-kube-api-access-nz4v9\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:30.627256 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:30.627120 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:31.453387 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:31.453351 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll" Jul 08 16:25:31.469082 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:31.469051 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:31.477163 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:31.477138 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5689eb-65fcf9dc87-pmmll"] Jul 08 16:25:33.268306 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:33.268272 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" path="/var/lib/kubelet/pods/8f7c2804-ea72-4f01-9d48-cdc9a6f7c969/volumes" Jul 08 16:25:34.284167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.284145 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:34.358012 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.357985 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358046 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358080 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358110 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358136 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358199 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358226 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358255 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9v6v\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358286 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:25:34.358376 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358332 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket\") pod \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\" (UID: \"55f7a7e3-55ce-42af-91d3-d419663c8cfe\") " Jul 08 16:25:34.358653 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358493 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:25:34.358653 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358521 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:25:34.358783 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358763 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:25:34.359112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358897 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data" (OuterVolumeSpecName: "istio-data") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:25:34.359112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.358940 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.359112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.359011 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.359112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.359024 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.360460 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.360434 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:25:34.360551 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.360508 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token" (OuterVolumeSpecName: "istio-token") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:25:34.360887 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.360871 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:25:34.361068 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.361054 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v" (OuterVolumeSpecName: "kube-api-access-q9v6v") pod "55f7a7e3-55ce-42af-91d3-d419663c8cfe" (UID: "55f7a7e3-55ce-42af-91d3-d419663c8cfe"). InnerVolumeSpecName "kube-api-access-q9v6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:25:34.459515 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459485 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.459515 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459511 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.459515 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459520 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.459706 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459528 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9v6v\" (UniqueName: \"kubernetes.io/projected/55f7a7e3-55ce-42af-91d3-d419663c8cfe-kube-api-access-q9v6v\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.459706 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459538 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.459706 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.459546 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f7a7e3-55ce-42af-91d3-d419663c8cfe-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:25:34.474247 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.474220 2566 generic.go:356] "Generic (PLEG): container finished" podID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerID="10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012" exitCode=0 Jul 08 16:25:34.474346 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.474274 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" Jul 08 16:25:34.474346 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.474297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerDied","Data":"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012"} Jul 08 16:25:34.474432 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.474348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f" event={"ID":"55f7a7e3-55ce-42af-91d3-d419663c8cfe","Type":"ContainerDied","Data":"32f41c4ef0702dfb9155e82fdacd1c37ca521789dfb3c895822ffaf79464a6eb"} Jul 08 16:25:34.474432 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.474369 2566 scope.go:117] "RemoveContainer" containerID="10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012" Jul 08 16:25:34.483966 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.483951 2566 scope.go:117] "RemoveContainer" containerID="8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a" Jul 08 16:25:34.492107 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.492087 2566 scope.go:117] "RemoveContainer" containerID="10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012" Jul 08 16:25:34.492341 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:25:34.492322 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012\": container with ID starting with 10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012 not found: ID does not exist" containerID="10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012" Jul 08 16:25:34.492391 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.492349 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012"} err="failed to get container status \"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012\": rpc error: code = NotFound desc = could not find container \"10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012\": container with ID starting with 10c7b4ddfa7243dd7fa718af8af3205cc00e3a8a32fdd3704b0ef292fa013012 not found: ID does not exist" Jul 08 16:25:34.492391 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.492365 2566 scope.go:117] "RemoveContainer" containerID="8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a" Jul 08 16:25:34.492618 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:25:34.492599 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a\": container with ID starting with 8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a not found: ID does not exist" containerID="8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a" Jul 08 16:25:34.492678 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.492627 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a"} err="failed to get container status \"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a\": rpc error: code = NotFound desc = could not find container \"8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a\": container with ID starting with 8636adfbc76de2f871d34bad06a5bb9e0c71d34a5154e8d07fd0a92fd69c7f4a not found: ID does not exist" Jul 08 16:25:34.498430 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.498406 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:25:34.506378 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:34.506359 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a3f98e-openshift-default-f78ddcff-wpk6f"] Jul 08 16:25:35.268973 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:35.268942 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" path="/var/lib/kubelet/pods/55f7a7e3-55ce-42af-91d3-d419663c8cfe/volumes" Jul 08 16:25:37.437660 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437631 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437953 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" containerName="maas-api" Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437963 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" containerName="maas-api" Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437976 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437981 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437993 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" containerName="maas-api" Jul 08 16:25:37.438008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.437999 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" containerName="maas-api" Jul 08 16:25:37.438206 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.438051 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f91fa5-19d3-4cb0-bdd2-c7fd8244c290" containerName="maas-api" Jul 08 16:25:37.438206 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.438062 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:37.438206 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.438068 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7c2804-ea72-4f01-9d48-cdc9a6f7c969" containerName="maas-api" Jul 08 16:25:37.438206 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.438074 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:37.441380 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.441358 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.444773 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.444752 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d4af16-openshift-default-dockercfg-8rqh7\"" Jul 08 16:25:37.473735 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.473712 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:25:37.582333 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582437 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65wt6\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582684 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582684 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.582684 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.582601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683828 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65wt6\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.683985 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.683971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684297 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684297 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684297 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684271 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684444 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684388 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684444 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684411 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684558 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684490 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.684665 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.684646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.686350 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.686326 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.686687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.686668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.692790 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.692731 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65wt6\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.692790 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.692764 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token\") pod \"e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.751467 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.751446 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:37.890960 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.890925 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:25:37.892734 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:25:37.892705 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb54909_b59c_424c_8350_ec01559951b5.slice/crio-927859f4ca334d7bc224aa318faae363e7a26dba33c53e6c1f20bc5571c8c844 WatchSource:0}: Error finding container 927859f4ca334d7bc224aa318faae363e7a26dba33c53e6c1f20bc5571c8c844: Status 404 returned error can't find the container with id 927859f4ca334d7bc224aa318faae363e7a26dba33c53e6c1f20bc5571c8c844 Jul 08 16:25:37.895170 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.895139 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:37.895253 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.895201 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:37.895253 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:37.895232 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:38.496918 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:38.496883 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerStarted","Data":"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141"} Jul 08 16:25:38.497317 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:38.496939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerStarted","Data":"927859f4ca334d7bc224aa318faae363e7a26dba33c53e6c1f20bc5571c8c844"} Jul 08 16:25:38.517905 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:38.517862 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podStartSLOduration=1.517846751 podStartE2EDuration="1.517846751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:25:38.516229528 +0000 UTC m=+2083.779248210" watchObservedRunningTime="2026-07-08 16:25:38.517846751 +0000 UTC m=+2083.780865420" Jul 08 16:25:38.752640 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:38.752523 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:38.757165 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:38.757140 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:39.502520 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:39.502488 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:39.503378 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:39.503355 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:40.625984 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.625958 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:25:40.626347 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.626302 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:40.626347 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.626313 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f7a7e3-55ce-42af-91d3-d419663c8cfe" containerName="istio-proxy" Jul 08 16:25:40.629161 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.629145 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.634955 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.634930 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4gvf8\"" Jul 08 16:25:40.635091 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.634989 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 16:25:40.635316 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.635299 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 16:25:40.635316 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.635308 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d4af16-serving-cert\"" Jul 08 16:25:40.669331 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.669305 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:25:40.712639 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.712605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gphkq\" (UniqueName: \"kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.712777 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.712666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.712777 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.712771 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.814216 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.814177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.814357 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.814242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gphkq\" (UniqueName: \"kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.814357 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.814277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.814968 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.814942 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.816740 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.816714 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.826765 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.826745 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gphkq\" (UniqueName: \"kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq\") pod \"maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:40.940359 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:40.940284 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:41.083256 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:41.083223 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:25:41.085031 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:25:41.085002 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f5a201_a123_4491_bbfe_1b563fe7538a.slice/crio-b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8 WatchSource:0}: Error finding container b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8: Status 404 returned error can't find the container with id b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8 Jul 08 16:25:41.518146 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:41.518105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" event={"ID":"32f5a201-a123-4491-bbfe-1b563fe7538a","Type":"ContainerStarted","Data":"b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8"} Jul 08 16:25:42.524491 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:42.524456 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" event={"ID":"32f5a201-a123-4491-bbfe-1b563fe7538a","Type":"ContainerStarted","Data":"20a1797e370bdba1645bc6300a52aa4e762bf16b56a445430fd3421890716bd7"} Jul 08 16:25:42.524983 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:42.524618 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:42.549112 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:42.549056 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" podStartSLOduration=1.839714672 podStartE2EDuration="2.549038594s" totalImagesPullingTime="709.323922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:25:42.548200025 +0000 UTC m=+2087.811218703" watchObservedRunningTime="2026-07-08 16:25:42.549038594 +0000 UTC m=+2087.812057264" Jul 08 16:25:48.537652 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:48.537621 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:25:49.396846 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.396808 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:25:49.400586 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.400542 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.403033 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.403011 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f77711-serving-cert\"" Jul 08 16:25:49.437591 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.436836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:25:49.488340 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.488309 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.488495 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.488392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.488495 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.488419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-659gh\" (UniqueName: \"kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.589028 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.588998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.589403 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.589042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-659gh\" (UniqueName: \"kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.589403 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.589076 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.589793 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.589774 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.591490 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.591472 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.598224 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.598197 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-659gh\" (UniqueName: \"kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh\") pod \"maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.710901 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.710825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:49.887671 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:49.887644 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:25:49.889624 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:25:49.889598 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d17efd_f85b_4ae0_a081_cebc33975b52.slice/crio-2cd0bf780b4f6148838597786bcf273a3b356e05008041f82341da704bae7d5f WatchSource:0}: Error finding container 2cd0bf780b4f6148838597786bcf273a3b356e05008041f82341da704bae7d5f: Status 404 returned error can't find the container with id 2cd0bf780b4f6148838597786bcf273a3b356e05008041f82341da704bae7d5f Jul 08 16:25:50.572715 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:50.572681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" event={"ID":"a1d17efd-f85b-4ae0-a081-cebc33975b52","Type":"ContainerStarted","Data":"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8"} Jul 08 16:25:50.572945 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:50.572733 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" event={"ID":"a1d17efd-f85b-4ae0-a081-cebc33975b52","Type":"ContainerStarted","Data":"2cd0bf780b4f6148838597786bcf273a3b356e05008041f82341da704bae7d5f"} Jul 08 16:25:50.572945 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:50.572800 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:54.504005 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.503941 2566 patch_prober.go:28] interesting pod/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 16:25:54.504339 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.503996 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 16:25:54.596298 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.596241 2566 generic.go:356] "Generic (PLEG): container finished" podID="1eb54909-b59c-424c-8350-ec01559951b5" containerID="0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141" exitCode=0 Jul 08 16:25:54.596420 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.596314 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerDied","Data":"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141"} Jul 08 16:25:54.596705 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.596690 2566 scope.go:117] "RemoveContainer" containerID="0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141" Jul 08 16:25:54.597689 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.597664 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:54.597758 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.597728 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:54.597799 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.597765 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 16:25:54.620462 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:54.620422 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" podStartSLOduration=5.079362357 podStartE2EDuration="5.620409711s" totalImagesPullingTime="541.047354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:25:50.590469995 +0000 UTC m=+2095.853488678" watchObservedRunningTime="2026-07-08 16:25:54.620409711 +0000 UTC m=+2099.883428399" Jul 08 16:25:55.329951 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.329922 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:25:55.335641 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.335616 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:25:55.337415 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.337392 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:25:55.342674 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.342659 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:25:55.603733 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.603705 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerStarted","Data":"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325"} Jul 08 16:25:55.751643 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.751602 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:55.751831 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.751810 2566 patch_prober.go:28] interesting pod/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 16:25:55.751905 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:55.751859 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 16:25:56.584918 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:56.584893 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:25:56.752336 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:56.752300 2566 patch_prober.go:28] interesting pod/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 16:25:56.752817 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:56.752365 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 16:25:57.752237 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:57.752198 2566 patch_prober.go:28] interesting pod/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 16:25:57.752627 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:57.752255 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 16:25:58.752767 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:58.752731 2566 patch_prober.go:28] interesting pod/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 16:25:58.753168 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:58.752782 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 16:25:59.755377 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:59.755346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:59.755881 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:59.755758 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:25:59.756297 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:25:59.756278 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:30:55.363765 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:30:55.363732 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:30:55.370167 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:30:55.370146 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:30:55.373485 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:30:55.373471 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:30:55.378657 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:30:55.378642 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:31:55.453004 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:31:55.452973 2566 scope.go:117] "RemoveContainer" containerID="2fd9caa61ca2b93630c73d469f5e0f1d367a95038ffb0ebe47f41940ea7a7ce1" Jul 08 16:32:11.315909 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.315868 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:32:11.316286 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.316051 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" podUID="32f5a201-a123-4491-bbfe-1b563fe7538a" containerName="maas-api" containerID="cri-o://20a1797e370bdba1645bc6300a52aa4e762bf16b56a445430fd3421890716bd7" gracePeriod=30 Jul 08 16:32:11.552499 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.552455 2566 generic.go:356] "Generic (PLEG): container finished" podID="32f5a201-a123-4491-bbfe-1b563fe7538a" containerID="20a1797e370bdba1645bc6300a52aa4e762bf16b56a445430fd3421890716bd7" exitCode=0 Jul 08 16:32:11.552649 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.552524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" event={"ID":"32f5a201-a123-4491-bbfe-1b563fe7538a","Type":"ContainerDied","Data":"20a1797e370bdba1645bc6300a52aa4e762bf16b56a445430fd3421890716bd7"} Jul 08 16:32:11.552714 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.552645 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" event={"ID":"32f5a201-a123-4491-bbfe-1b563fe7538a","Type":"ContainerDied","Data":"b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8"} Jul 08 16:32:11.552714 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.552675 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0241bec716c97c1efa63280e49e3af15a494c82649fdd107dbff8c47b50b0d8" Jul 08 16:32:11.567219 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.567169 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:32:11.574154 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.574134 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca\") pod \"32f5a201-a123-4491-bbfe-1b563fe7538a\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " Jul 08 16:32:11.574263 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.574215 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gphkq\" (UniqueName: \"kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq\") pod \"32f5a201-a123-4491-bbfe-1b563fe7538a\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " Jul 08 16:32:11.574263 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.574247 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls\") pod \"32f5a201-a123-4491-bbfe-1b563fe7538a\" (UID: \"32f5a201-a123-4491-bbfe-1b563fe7538a\") " Jul 08 16:32:11.574453 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.574428 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32f5a201-a123-4491-bbfe-1b563fe7538a" (UID: "32f5a201-a123-4491-bbfe-1b563fe7538a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:32:11.576412 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.576386 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq" (OuterVolumeSpecName: "kube-api-access-gphkq") pod "32f5a201-a123-4491-bbfe-1b563fe7538a" (UID: "32f5a201-a123-4491-bbfe-1b563fe7538a"). InnerVolumeSpecName "kube-api-access-gphkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:32:11.576412 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.576391 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32f5a201-a123-4491-bbfe-1b563fe7538a" (UID: "32f5a201-a123-4491-bbfe-1b563fe7538a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:32:11.674919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.674884 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32f5a201-a123-4491-bbfe-1b563fe7538a-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:11.674919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.674913 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gphkq\" (UniqueName: \"kubernetes.io/projected/32f5a201-a123-4491-bbfe-1b563fe7538a-kube-api-access-gphkq\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:11.674919 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:11.674926 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32f5a201-a123-4491-bbfe-1b563fe7538a-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:12.557105 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:12.557071 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9" Jul 08 16:32:12.578160 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:12.578135 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:32:12.582106 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:12.582088 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d4af16-5bf8dd56b-92cj9"] Jul 08 16:32:13.269086 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:13.269057 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32f5a201-a123-4491-bbfe-1b563fe7538a" path="/var/lib/kubelet/pods/32f5a201-a123-4491-bbfe-1b563fe7538a/volumes" Jul 08 16:32:17.528710 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:17.528677 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:32:17.529115 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:17.528871 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" containerID="cri-o://5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325" gracePeriod=30 Jul 08 16:32:18.687317 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:18.687281 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:32:18.687792 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:18.687482 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" podUID="a1d17efd-f85b-4ae0-a081-cebc33975b52" containerName="maas-api" containerID="cri-o://6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8" gracePeriod=30 Jul 08 16:32:18.933151 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:18.933127 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:32:19.028355 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.028269 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-659gh\" (UniqueName: \"kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh\") pod \"a1d17efd-f85b-4ae0-a081-cebc33975b52\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " Jul 08 16:32:19.028355 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.028300 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls\") pod \"a1d17efd-f85b-4ae0-a081-cebc33975b52\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " Jul 08 16:32:19.028355 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.028340 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca\") pod \"a1d17efd-f85b-4ae0-a081-cebc33975b52\" (UID: \"a1d17efd-f85b-4ae0-a081-cebc33975b52\") " Jul 08 16:32:19.028778 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.028749 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1d17efd-f85b-4ae0-a081-cebc33975b52" (UID: "a1d17efd-f85b-4ae0-a081-cebc33975b52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:32:19.030626 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.030595 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1d17efd-f85b-4ae0-a081-cebc33975b52" (UID: "a1d17efd-f85b-4ae0-a081-cebc33975b52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:32:19.030725 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.030627 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh" (OuterVolumeSpecName: "kube-api-access-659gh") pod "a1d17efd-f85b-4ae0-a081-cebc33975b52" (UID: "a1d17efd-f85b-4ae0-a081-cebc33975b52"). InnerVolumeSpecName "kube-api-access-659gh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:32:19.128853 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.128825 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-659gh\" (UniqueName: \"kubernetes.io/projected/a1d17efd-f85b-4ae0-a081-cebc33975b52-kube-api-access-659gh\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:19.128853 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.128849 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d17efd-f85b-4ae0-a081-cebc33975b52-maas-api-tls\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:19.129010 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.128860 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d17efd-f85b-4ae0-a081-cebc33975b52-openshift-service-ca\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:19.593653 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.593621 2566 generic.go:356] "Generic (PLEG): container finished" podID="a1d17efd-f85b-4ae0-a081-cebc33975b52" containerID="6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8" exitCode=0 Jul 08 16:32:19.593852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.593662 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" Jul 08 16:32:19.593852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.593715 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" event={"ID":"a1d17efd-f85b-4ae0-a081-cebc33975b52","Type":"ContainerDied","Data":"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8"} Jul 08 16:32:19.593852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.593768 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9" event={"ID":"a1d17efd-f85b-4ae0-a081-cebc33975b52","Type":"ContainerDied","Data":"2cd0bf780b4f6148838597786bcf273a3b356e05008041f82341da704bae7d5f"} Jul 08 16:32:19.593852 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.593798 2566 scope.go:117] "RemoveContainer" containerID="6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8" Jul 08 16:32:19.602840 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.602823 2566 scope.go:117] "RemoveContainer" containerID="6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8" Jul 08 16:32:19.603094 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:32:19.603072 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8\": container with ID starting with 6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8 not found: ID does not exist" containerID="6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8" Jul 08 16:32:19.603200 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.603097 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8"} err="failed to get container status \"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8\": rpc error: code = NotFound desc = could not find container \"6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8\": container with ID starting with 6fff80a5fadb95b07ec526a8ba87c673d0400512b4becae9400a49e680809dc8 not found: ID does not exist" Jul 08 16:32:19.611056 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.611033 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:32:19.616907 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:19.616805 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f77711-7698d57b74-vtgs9"] Jul 08 16:32:21.269036 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:21.269002 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1d17efd-f85b-4ae0-a081-cebc33975b52" path="/var/lib/kubelet/pods/a1d17efd-f85b-4ae0-a081-cebc33975b52/volumes" Jul 08 16:32:22.786116 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.786090 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:32:22.857972 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.857947 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858124 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.857979 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858124 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.857999 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858124 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858060 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858124 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858108 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858134 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65wt6\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858221 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858253 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket\") pod \"1eb54909-b59c-424c-8350-ec01559951b5\" (UID: \"1eb54909-b59c-424c-8350-ec01559951b5\") " Jul 08 16:32:22.858370 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858260 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:32:22.858683 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858527 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-credential-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.858683 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858532 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:32:22.858683 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858602 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:32:22.859008 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.858738 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:32:22.859194 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.859164 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:32:22.860437 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.860405 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 16:32:22.860437 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.860413 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:32:22.860661 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.860478 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6" (OuterVolumeSpecName: "kube-api-access-65wt6") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "kube-api-access-65wt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:32:22.860661 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.860518 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "1eb54909-b59c-424c-8350-ec01559951b5" (UID: "1eb54909-b59c-424c-8350-ec01559951b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:32:22.959802 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959774 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1eb54909-b59c-424c-8350-ec01559951b5-istio-podinfo\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959802 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959796 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-envoy\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959802 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959805 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-istio-token\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959996 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959813 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-certs\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959996 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959821 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-istio-data\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959996 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959829 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65wt6\" (UniqueName: \"kubernetes.io/projected/1eb54909-b59c-424c-8350-ec01559951b5-kube-api-access-65wt6\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959996 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959837 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1eb54909-b59c-424c-8350-ec01559951b5-istiod-ca-cert\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:22.959996 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:22.959845 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1eb54909-b59c-424c-8350-ec01559951b5-workload-socket\") on node \"ip-10-0-136-198.ec2.internal\" DevicePath \"\"" Jul 08 16:32:23.615238 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.615200 2566 generic.go:356] "Generic (PLEG): container finished" podID="1eb54909-b59c-424c-8350-ec01559951b5" containerID="5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325" exitCode=0 Jul 08 16:32:23.615428 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.615253 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" Jul 08 16:32:23.615428 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.615291 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerDied","Data":"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325"} Jul 08 16:32:23.615428 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.615348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz" event={"ID":"1eb54909-b59c-424c-8350-ec01559951b5","Type":"ContainerDied","Data":"927859f4ca334d7bc224aa318faae363e7a26dba33c53e6c1f20bc5571c8c844"} Jul 08 16:32:23.615428 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.615376 2566 scope.go:117] "RemoveContainer" containerID="5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325" Jul 08 16:32:23.624178 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.624158 2566 scope.go:117] "RemoveContainer" containerID="0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141" Jul 08 16:32:23.631747 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.631731 2566 scope.go:117] "RemoveContainer" containerID="5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325" Jul 08 16:32:23.632000 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:32:23.631964 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325\": container with ID starting with 5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325 not found: ID does not exist" containerID="5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325" Jul 08 16:32:23.632085 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.632001 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325"} err="failed to get container status \"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325\": rpc error: code = NotFound desc = could not find container \"5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325\": container with ID starting with 5bdae676053a9abba5e3b5dd73c47b5909c7cd8587144b10fb214c242b8de325 not found: ID does not exist" Jul 08 16:32:23.632085 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.632025 2566 scope.go:117] "RemoveContainer" containerID="0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141" Jul 08 16:32:23.632298 ip-10-0-136-198 kubenswrapper[2566]: E0708 16:32:23.632280 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141\": container with ID starting with 0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141 not found: ID does not exist" containerID="0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141" Jul 08 16:32:23.632344 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.632303 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141"} err="failed to get container status \"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141\": rpc error: code = NotFound desc = could not find container \"0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141\": container with ID starting with 0c3e2663a35a8970c6c061b2fee2e392057a551b39f072aafe77973df2325141 not found: ID does not exist" Jul 08 16:32:23.648497 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.648477 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:32:23.654783 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:23.654764 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d4af16-openshift-default-5f66cb48cf-nchvz"] Jul 08 16:32:25.269320 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:25.269289 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eb54909-b59c-424c-8350-ec01559951b5" path="/var/lib/kubelet/pods/1eb54909-b59c-424c-8350-ec01559951b5/volumes" Jul 08 16:32:30.878719 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:30.878687 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c94cb7c4-j6k98_cc22f47f-0e1b-4340-9ee2-436085a849aa/copy-manifests/0.log" Jul 08 16:32:30.885966 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:30.885939 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c94cb7c4-j6k98_cc22f47f-0e1b-4340-9ee2-436085a849aa/manager/0.log" Jul 08 16:32:31.651486 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.651450 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/extract/0.log" Jul 08 16:32:31.657759 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.657731 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/util/0.log" Jul 08 16:32:31.664105 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.664080 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/pull/0.log" Jul 08 16:32:31.785528 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.785505 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/util/0.log" Jul 08 16:32:31.791459 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.791429 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/pull/0.log" Jul 08 16:32:31.797264 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.797229 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/extract/0.log" Jul 08 16:32:31.906992 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.906940 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/util/0.log" Jul 08 16:32:31.913960 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.913940 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/pull/0.log" Jul 08 16:32:31.920511 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:31.920495 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/extract/0.log" Jul 08 16:32:32.044253 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:32.044225 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/extract/0.log" Jul 08 16:32:32.050188 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:32.050165 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/util/0.log" Jul 08 16:32:32.056310 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:32.056270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/pull/0.log" Jul 08 16:32:33.002204 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:33.002166 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rhbgr_218f8f58-d8be-4693-ad0a-145d7d5a26da/manager/0.log" Jul 08 16:32:33.481584 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:33.481537 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5t6g5_dcf49aaa-ecaa-4172-9f78-dbf73ea7e612/discovery/0.log" Jul 08 16:32:34.168847 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:34.168799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdcbbdb9-r5lfj_41e492fc-181e-4715-8fc9-26e02d17b789/router/0.log" Jul 08 16:32:34.520740 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:34.520651 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4f256_a4c1b4f6-e94c-45d6-a508-d0e8fa3befea/main/0.log" Jul 08 16:32:34.984313 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:34.984250 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-56mq9_da273f17-1924-4f7f-b612-180da4ee9fd1/main/0.log" Jul 08 16:32:42.275064 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:42.275034 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tczlz_48789b43-33de-4682-aeb5-06bbde0259e9/global-pull-secret-syncer/0.log" Jul 08 16:32:42.391643 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:42.391614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pshm9_41d09967-e2a6-4950-83dc-b5a41917b035/konnectivity-agent/0.log" Jul 08 16:32:42.457647 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:42.457616 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-198.ec2.internal_4bb0963c382b2442e093109581c52883/haproxy/0.log" Jul 08 16:32:46.833389 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.833300 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/extract/0.log" Jul 08 16:32:46.857244 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.857220 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/util/0.log" Jul 08 16:32:46.878273 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.878243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztsl8_cf7b3505-7b06-4171-8e1d-5c89caea98b3/pull/0.log" Jul 08 16:32:46.907677 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.907657 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/extract/0.log" Jul 08 16:32:46.932429 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.932407 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/util/0.log" Jul 08 16:32:46.957048 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.957025 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fr97d_9a2e37d4-2f38-466e-997b-b0ea9382fbb1/pull/0.log" Jul 08 16:32:46.984058 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:46.984034 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/extract/0.log" Jul 08 16:32:47.011001 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.010978 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/util/0.log" Jul 08 16:32:47.035348 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.035327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p67tk_a9f0b268-389a-437c-9ea6-29e7750b9553/pull/0.log" Jul 08 16:32:47.066317 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.066298 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/extract/0.log" Jul 08 16:32:47.091721 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.091673 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/util/0.log" Jul 08 16:32:47.124159 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.124126 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12sslt_027d33d0-e0d8-475b-948a-c31fc3bb95b7/pull/0.log" Jul 08 16:32:47.978152 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:47.978123 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rhbgr_218f8f58-d8be-4693-ad0a-145d7d5a26da/manager/0.log" Jul 08 16:32:49.734398 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:49.734364 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-lf9rm_82e36f17-b33c-406c-9648-b0078e7c4184/cluster-monitoring-operator/0.log" Jul 08 16:32:49.842042 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:49.842020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65cdc46999-hqkvm_f9e767ce-0def-4b36-967d-0f72f0314e79/metrics-server/0.log" Jul 08 16:32:50.109407 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:50.109378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx22t_f32eb20b-3c70-4905-9780-0b3d25b6e806/node-exporter/0.log" Jul 08 16:32:50.129597 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:50.129554 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx22t_f32eb20b-3c70-4905-9780-0b3d25b6e806/kube-rbac-proxy/0.log" Jul 08 16:32:50.155311 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:50.155295 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx22t_f32eb20b-3c70-4905-9780-0b3d25b6e806/init-textfile/0.log" Jul 08 16:32:50.449484 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:50.449404 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5d754_0f943243-e356-4d94-8635-709576aa00e6/prometheus-operator/0.log" Jul 08 16:32:50.468624 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:50.468598 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5d754_0f943243-e356-4d94-8635-709576aa00e6/kube-rbac-proxy/0.log" Jul 08 16:32:51.972978 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:51.972949 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-g6nhn_0116e6b5-d804-4c43-a6f6-931f53fe446c/networking-console-plugin/0.log" Jul 08 16:32:52.576556 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.576527 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/1.log" Jul 08 16:32:52.585305 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.585225 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7j7sj_0a33585d-426f-43fc-98c8-2fd6a1e00fef/console-operator/2.log" Jul 08 16:32:52.831878 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.831796 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn"] Jul 08 16:32:52.832223 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832206 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32f5a201-a123-4491-bbfe-1b563fe7538a" containerName="maas-api" Jul 08 16:32:52.832223 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832226 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32f5a201-a123-4491-bbfe-1b563fe7538a" containerName="maas-api" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832243 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832252 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832269 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832275 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832284 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d17efd-f85b-4ae0-a081-cebc33975b52" containerName="maas-api" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832292 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d17efd-f85b-4ae0-a081-cebc33975b52" containerName="maas-api" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832352 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832365 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb54909-b59c-424c-8350-ec01559951b5" containerName="istio-proxy" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832377 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d17efd-f85b-4ae0-a081-cebc33975b52" containerName="maas-api" Jul 08 16:32:52.832383 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.832383 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32f5a201-a123-4491-bbfe-1b563fe7538a" containerName="maas-api" Jul 08 16:32:52.835619 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.835601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.842974 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.838165 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7sqg6\"/\"kube-root-ca.crt\"" Jul 08 16:32:52.842974 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.838909 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7sqg6\"/\"openshift-service-ca.crt\"" Jul 08 16:32:52.842974 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.839137 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7sqg6\"/\"default-dockercfg-89vfp\"" Jul 08 16:32:52.847746 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.847721 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn"] Jul 08 16:32:52.888856 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.888832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-proc\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.888971 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.888868 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-sys\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.888971 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.888887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-podres\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.888971 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.888941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-lib-modules\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.889078 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.888987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk9s\" (UniqueName: \"kubernetes.io/projected/e49fc0cf-8f54-45d1-958f-280d311330e6-kube-api-access-mxk9s\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989687 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-proc\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-sys\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-podres\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-lib-modules\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989785 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-proc\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989784 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk9s\" (UniqueName: \"kubernetes.io/projected/e49fc0cf-8f54-45d1-958f-280d311330e6-kube-api-access-mxk9s\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989788 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-sys\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989859 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-podres\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.989994 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.989872 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e49fc0cf-8f54-45d1-958f-280d311330e6-lib-modules\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:52.998039 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:52.998014 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk9s\" (UniqueName: \"kubernetes.io/projected/e49fc0cf-8f54-45d1-958f-280d311330e6-kube-api-access-mxk9s\") pod \"perf-node-gather-daemonset-pg9qn\" (UID: \"e49fc0cf-8f54-45d1-958f-280d311330e6\") " pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:53.148872 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.148850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:53.282736 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.282713 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn"] Jul 08 16:32:53.284295 ip-10-0-136-198 kubenswrapper[2566]: W0708 16:32:53.284266 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode49fc0cf_8f54_45d1_958f_280d311330e6.slice/crio-41d73a970a23d18fbbac8d7d94b9a147da4cabc83ddbbae50f3ef1ce645f16eb WatchSource:0}: Error finding container 41d73a970a23d18fbbac8d7d94b9a147da4cabc83ddbbae50f3ef1ce645f16eb: Status 404 returned error can't find the container with id 41d73a970a23d18fbbac8d7d94b9a147da4cabc83ddbbae50f3ef1ce645f16eb Jul 08 16:32:53.764955 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.764866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" event={"ID":"e49fc0cf-8f54-45d1-958f-280d311330e6","Type":"ContainerStarted","Data":"a99ff4889f271b9715fec765ebdd377d5fe6b9711164e68b8b3be80554e4cc0a"} Jul 08 16:32:53.764955 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.764915 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" event={"ID":"e49fc0cf-8f54-45d1-958f-280d311330e6","Type":"ContainerStarted","Data":"41d73a970a23d18fbbac8d7d94b9a147da4cabc83ddbbae50f3ef1ce645f16eb"} Jul 08 16:32:53.765225 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.764985 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:32:53.781641 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.781564 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" podStartSLOduration=1.7815503750000001 podStartE2EDuration="1.781550375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:32:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:32:53.780239258 +0000 UTC m=+2519.043257928" watchObservedRunningTime="2026-07-08 16:32:53.781550375 +0000 UTC m=+2519.044569044" Jul 08 16:32:53.818870 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:53.818844 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-cmwrw_09287bf3-d83b-445e-a731-2e541313c740/volume-data-source-validator/0.log" Jul 08 16:32:54.882804 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:54.882770 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c79v7_7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9/dns/0.log" Jul 08 16:32:54.904438 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:54.904414 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c79v7_7e5dd1c8-44a9-4fcd-88eb-e3c32674ddf9/kube-rbac-proxy/0.log" Jul 08 16:32:54.999897 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:54.999871 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h6n7j_8ead0a15-65c5-47a6-b1b6-10bdf1b92031/dns-node-resolver/0.log" Jul 08 16:32:55.473155 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:55.473127 2566 scope.go:117] "RemoveContainer" containerID="20a1797e370bdba1645bc6300a52aa4e762bf16b56a445430fd3421890716bd7" Jul 08 16:32:55.647418 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:55.647386 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-drs65_91a472ac-8359-42de-8fb2-3fae3c6117c0/node-ca/0.log" Jul 08 16:32:56.808173 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:56.808143 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5t6g5_dcf49aaa-ecaa-4172-9f78-dbf73ea7e612/discovery/0.log" Jul 08 16:32:57.018945 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:57.018913 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdcbbdb9-r5lfj_41e492fc-181e-4715-8fc9-26e02d17b789/router/0.log" Jul 08 16:32:57.725794 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:57.725763 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8ppj7_11b7be65-4a8b-44e0-8177-b8b117407273/serve-healthcheck-canary/0.log" Jul 08 16:32:58.279286 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:58.279251 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-klzjd_9ee637f4-2915-4fd8-9ce8-58d4629dbf65/insights-operator/0.log" Jul 08 16:32:58.280324 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:58.280304 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-klzjd_9ee637f4-2915-4fd8-9ce8-58d4629dbf65/insights-operator/1.log" Jul 08 16:32:58.302876 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:58.302855 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7w2mq_6142acdd-6d9e-4979-aa6e-85ad14509660/kube-rbac-proxy/0.log" Jul 08 16:32:58.324910 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:58.324890 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7w2mq_6142acdd-6d9e-4979-aa6e-85ad14509660/exporter/0.log" Jul 08 16:32:58.346808 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:58.346786 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7w2mq_6142acdd-6d9e-4979-aa6e-85ad14509660/extractor/0.log" Jul 08 16:32:59.780117 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:32:59.780091 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7sqg6/perf-node-gather-daemonset-pg9qn" Jul 08 16:33:01.199849 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:01.199820 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c94cb7c4-j6k98_cc22f47f-0e1b-4340-9ee2-436085a849aa/manager/0.log" Jul 08 16:33:01.219102 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:01.219082 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c94cb7c4-j6k98_cc22f47f-0e1b-4340-9ee2-436085a849aa/copy-manifests/0.log" Jul 08 16:33:02.362086 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:02.362046 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57b64f8b44-884b2_4610c5db-c917-4822-bab3-36ae1c3b7d20/manager/0.log" Jul 08 16:33:02.994723 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:02.994687 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5t6g5_dcf49aaa-ecaa-4172-9f78-dbf73ea7e612/discovery/0.log" Jul 08 16:33:03.189260 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:03.189182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdcbbdb9-r5lfj_41e492fc-181e-4715-8fc9-26e02d17b789/router/0.log" Jul 08 16:33:04.702769 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:04.702736 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5t6g5_dcf49aaa-ecaa-4172-9f78-dbf73ea7e612/discovery/0.log" Jul 08 16:33:04.879054 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:04.879012 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdcbbdb9-r5lfj_41e492fc-181e-4715-8fc9-26e02d17b789/router/0.log" Jul 08 16:33:08.341592 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:08.341541 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wbc58_d6d0f68f-8bae-4925-afb4-bc71338d6599/kube-storage-version-migrator-operator/1.log" Jul 08 16:33:08.343235 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:08.343203 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wbc58_d6d0f68f-8bae-4925-afb4-bc71338d6599/kube-storage-version-migrator-operator/0.log" Jul 08 16:33:09.500026 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.499995 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/kube-multus-additional-cni-plugins/0.log" Jul 08 16:33:09.523102 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.523075 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/egress-router-binary-copy/0.log" Jul 08 16:33:09.544285 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.544263 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/cni-plugins/0.log" Jul 08 16:33:09.564076 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.564053 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/bond-cni-plugin/0.log" Jul 08 16:33:09.583481 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.583460 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/routeoverride-cni/0.log" Jul 08 16:33:09.605973 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.605956 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/whereabouts-cni-bincopy/0.log" Jul 08 16:33:09.626029 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.626015 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pf49f_c84d0b88-ac06-40b3-b61c-9cfa46f1aad4/whereabouts-cni/0.log" Jul 08 16:33:09.871709 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.871682 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-htrtz_2a96e133-d26d-4fa8-827c-6dae2afe9e64/kube-multus/0.log" Jul 08 16:33:09.927314 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.927286 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-772cg_0ac03da3-cc5b-498c-b834-11719d2f4a21/network-metrics-daemon/0.log" Jul 08 16:33:09.944685 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:09.944632 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-772cg_0ac03da3-cc5b-498c-b834-11719d2f4a21/kube-rbac-proxy/0.log" Jul 08 16:33:11.087101 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.087062 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-controller/0.log" Jul 08 16:33:11.103820 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.103792 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/1.log" Jul 08 16:33:11.103984 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.103799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovn-acl-logging/0.log" Jul 08 16:33:11.126087 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.126072 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/kube-rbac-proxy-node/0.log" Jul 08 16:33:11.147342 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.147312 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 16:33:11.163517 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.163500 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/northd/0.log" Jul 08 16:33:11.183262 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.183246 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/nbdb/0.log" Jul 08 16:33:11.203407 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.203387 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/sbdb/0.log" Jul 08 16:33:11.284514 ip-10-0-136-198 kubenswrapper[2566]: I0708 16:33:11.284479 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5np9_a2109ac5-4292-4774-9731-c6dccb981dfc/ovnkube-controller/0.log"