Jul 15 19:58:27.013360 ip-10-0-135-159 systemd[1]: Starting Kubernetes Kubelet... Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.759306 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770104 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770117 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770120 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770124 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770127 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770130 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770133 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.784553 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770136 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770139 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770142 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770145 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770148 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770150 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770154 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770157 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770159 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770162 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770164 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770167 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770169 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770181 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770184 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770187 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770189 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770192 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770195 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770198 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.787066 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770200 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770203 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770205 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770208 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770211 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770214 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770217 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770219 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770222 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770224 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770227 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770230 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770232 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770235 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770238 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770241 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770245 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770249 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770252 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.787787 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770255 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770258 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770261 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770264 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770267 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770270 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770273 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770276 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770278 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770281 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770284 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770286 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770290 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770292 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770295 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770297 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770300 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770303 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770306 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770308 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.789648 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770311 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770314 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770316 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770319 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770323 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770326 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770329 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770332 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770336 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770339 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770342 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770345 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770347 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770350 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770352 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770355 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770357 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770359 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770364 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.790353 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770367 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770370 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770373 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770376 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770378 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770382 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770384 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770387 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770390 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770393 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770395 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770398 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770400 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.770403 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771699 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771707 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771711 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771714 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771717 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771723 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.793003 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771726 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771729 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771732 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771734 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771737 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771739 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771742 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771745 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771748 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771750 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771753 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771756 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771759 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771762 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771764 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771767 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771770 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771773 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771777 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.793689 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771780 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771783 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771786 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771789 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771792 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771794 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771797 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771799 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771802 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771805 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771807 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771809 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771812 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771815 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771817 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771820 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771822 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771825 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771828 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.796557 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771832 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771835 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771838 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771841 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771845 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771848 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771850 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771853 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771855 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771858 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771861 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771864 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771866 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771869 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771872 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771874 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771877 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771880 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771882 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771885 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.797174 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771888 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771890 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771893 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771896 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771898 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771901 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771904 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771907 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771910 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771912 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771915 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771917 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771920 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771923 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771926 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771928 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771931 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771933 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771935 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771938 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.797801 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771940 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771943 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771945 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771948 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771950 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771953 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771955 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771958 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771960 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771963 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771966 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771968 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771971 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771974 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.771978 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772373 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772379 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772382 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772385 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772389 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.800395 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772392 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772395 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772397 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772400 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772402 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772405 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772407 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772410 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772413 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772415 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772418 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772420 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772423 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772425 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772428 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772431 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772435 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772439 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772442 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772445 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.801036 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772451 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772453 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772456 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772458 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772461 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772463 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772466 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772469 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772472 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772475 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772478 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772480 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772484 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772486 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772489 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772492 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772494 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772497 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772500 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772502 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.801668 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772505 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772509 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772512 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772515 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772517 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772520 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772523 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772525 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772528 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772531 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772534 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772536 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772540 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772543 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772547 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772549 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772552 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772554 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772557 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.803428 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772559 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772562 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772565 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772567 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772570 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772573 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772576 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772579 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772582 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772584 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772586 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772589 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772592 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772596 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772598 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772601 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772603 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772606 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772608 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772611 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.804097 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772613 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772616 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772618 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772621 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772623 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772627 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772629 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772633 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772636 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772639 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772641 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772643 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772646 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772648 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.772651 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772723 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772729 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772735 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772740 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772744 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772747 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 19:58:27.804716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772752 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772756 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772759 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772762 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772765 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772768 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772771 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772774 2557 flags.go:64] FLAG: --cgroup-root="" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772777 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772780 2557 flags.go:64] FLAG: --client-ca-file="" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772783 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772786 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772791 2557 flags.go:64] FLAG: --cluster-domain="" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772793 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772796 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772799 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772805 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772813 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772817 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772820 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772823 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772826 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772829 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772832 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 19:58:27.859655 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772835 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772837 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772841 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772844 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772847 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772850 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772852 2557 flags.go:64] FLAG: --enable-server="true" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772856 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772861 2557 flags.go:64] FLAG: --event-burst="100" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772864 2557 flags.go:64] FLAG: --event-qps="50" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772866 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772869 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772872 2557 flags.go:64] FLAG: --eviction-hard="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772876 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772879 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772882 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772884 2557 flags.go:64] FLAG: --eviction-soft="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772887 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772890 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772893 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772896 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772898 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772901 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772904 2557 flags.go:64] FLAG: --feature-gates="" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772909 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 19:58:27.860459 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772912 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772917 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772920 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772923 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772926 2557 flags.go:64] FLAG: --help="false" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772929 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-135-159.ec2.internal" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772932 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772935 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772937 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772941 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772944 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772947 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772950 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772952 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772955 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772959 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772962 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772965 2557 flags.go:64] FLAG: --kube-reserved="" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772968 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772971 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772973 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772976 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772979 2557 flags.go:64] FLAG: --lock-file="" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772981 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 19:58:27.862733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772984 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772987 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772992 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772995 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.772997 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773000 2557 flags.go:64] FLAG: --logging-format="text" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773003 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773007 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773010 2557 flags.go:64] FLAG: --manifest-url="" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773014 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773018 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773021 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773026 2557 flags.go:64] FLAG: --max-pods="110" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773029 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773032 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773035 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773038 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773041 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773043 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773046 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773054 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773057 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773060 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 19:58:27.863488 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773063 2557 flags.go:64] FLAG: --pod-cidr="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773066 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773085 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773088 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773091 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773094 2557 flags.go:64] FLAG: --port="10250" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773097 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773100 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb7212a00106fbd4" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773103 2557 flags.go:64] FLAG: --qos-reserved="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773106 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773109 2557 flags.go:64] FLAG: --register-node="true" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773111 2557 flags.go:64] FLAG: --register-with-taints="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773115 2557 flags.go:64] FLAG: --registry-burst="10" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773118 2557 flags.go:64] FLAG: --registry-qps="5" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773120 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773123 2557 flags.go:64] FLAG: --reserved-memory="" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773128 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773131 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773134 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773138 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773141 2557 flags.go:64] FLAG: --runonce="false" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773143 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773146 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773149 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773152 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 19:58:27.938603 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773155 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 19:58:27.931815 ip-10-0-135-159 systemd[1]: Started Kubernetes Kubelet. Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773158 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773161 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773164 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773167 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773170 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773173 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773175 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773178 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773182 2557 flags.go:64] FLAG: --system-cgroups="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773184 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773190 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773193 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773196 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773200 2557 flags.go:64] FLAG: --tls-min-version="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773203 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773206 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773208 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773211 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773214 2557 flags.go:64] FLAG: --v="2" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773218 2557 flags.go:64] FLAG: --version="false" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773222 2557 flags.go:64] FLAG: --vmodule="" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773225 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.773229 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773361 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.939703 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773365 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773369 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773372 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773375 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773377 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773380 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773382 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773385 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773388 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773391 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773393 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773396 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773398 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773401 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773403 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773406 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773409 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773413 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773415 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773418 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.940638 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773420 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773423 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773425 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773428 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773430 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773432 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773435 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773437 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773440 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773442 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773446 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773449 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773458 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773463 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773465 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773468 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773470 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773473 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773476 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773478 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.941534 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773481 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773484 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773487 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773489 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773492 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773494 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773497 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773499 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773502 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773504 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773507 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773510 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773513 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773515 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773518 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773522 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773525 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773528 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773531 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.942429 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773533 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773536 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773538 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773542 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773544 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773548 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773553 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773556 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773559 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773561 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773564 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773567 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773570 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773574 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773576 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773579 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773582 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773584 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773586 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.943493 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773589 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773591 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773594 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773596 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773599 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773602 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773604 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773607 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773610 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773612 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773615 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773617 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773619 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773622 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773624 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773627 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773631 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773634 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773636 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.944140 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.773640 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.774516 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.781901 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.781916 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.781940 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.781948 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782024 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782030 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782035 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782040 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782044 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782048 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782052 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782056 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782060 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.944640 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782064 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782081 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782087 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782095 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782100 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782104 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782109 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782112 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782116 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782120 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782124 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782128 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782132 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782136 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782140 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782143 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782147 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782151 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782155 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782159 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.945197 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782164 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782167 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782171 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782175 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782179 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782183 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782187 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782191 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782194 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782198 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782202 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782205 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782209 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782213 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782216 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782219 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782223 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782227 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782232 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782235 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.945673 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782239 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782243 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782247 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782251 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782255 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782259 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782262 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782266 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782269 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782273 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782276 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782284 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782288 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782292 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782296 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782299 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782303 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782307 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782310 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.946210 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782316 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782321 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782326 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782329 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782334 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782337 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782341 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782346 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782350 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782354 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782358 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782362 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782367 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782371 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782375 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782379 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782383 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782387 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782391 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782395 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.946757 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782400 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782403 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782407 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782411 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782418 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782422 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782426 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782431 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782436 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782440 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782445 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.782452 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782602 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782608 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:58:27.947725 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782613 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782617 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782621 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782624 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782628 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782632 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782636 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782640 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782644 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782648 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782651 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782655 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782659 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782664 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782669 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782673 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782677 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782681 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782684 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782687 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:58:27.948311 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782691 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782695 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782700 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782706 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782709 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782714 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782718 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782722 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782726 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782729 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782732 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782736 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782740 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782744 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782748 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782751 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782755 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782759 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782763 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782767 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:58:27.948950 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782771 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782774 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782779 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782782 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782786 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782789 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782793 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782796 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782800 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782804 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782807 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782810 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782814 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782818 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782823 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782829 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782832 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782836 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782840 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782844 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:58:27.949831 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782848 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782853 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782858 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782862 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782866 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782870 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782874 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782878 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782882 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782886 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782889 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782893 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782897 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782901 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782904 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782908 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782912 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782916 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782919 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:58:27.950527 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782922 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782928 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782932 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782936 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782941 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782945 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782949 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782955 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782960 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782963 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782967 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782971 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782974 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782978 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782982 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782985 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782989 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:58:27.950949 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:27.782992 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.782999 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.783135 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.859562 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.860908 2557 server.go:1019] "Starting client certificate rotation" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.861010 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.861048 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.863245 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.867002 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.875808 2557 log.go:25] "Validated CRI v1 runtime API" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.877633 2557 log.go:25] "Validated CRI v1 image API" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.889591 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:58:27.951400 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.903808 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 19:58:27.951683 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.908374 2557 fs.go:135] Filesystem UUIDs: map[2ce91607-16f4-46a6-a078-8d123827fed1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d5ada51c-db81-4c9a-ade5-11c7566541d3:/dev/nvme0n1p3] Jul 15 19:58:27.951683 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.908397 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.914337 2557 manager.go:216] Machine: {Timestamp:2026-07-15 19:58:27.912741394 +0000 UTC m=+0.784734816 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099767 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25f54a72bd50fd6ec17b0971d407e8 SystemUUID:ec25f54a-72bd-50fd-6ec1-7b0971d407e8 BootID:6d3c403a-18b9-48a8-b889-cf32147b2d5b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:71:ce:4c:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:71:ce:4c:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:10:aa:12:74:b8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.914460 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.914587 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.916434 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.916454 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.917663 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.917673 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.917680 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.918333 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.919473 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922257 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922356 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922381 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922389 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922398 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922406 2557 kubelet.go:398] "Adding apiserver pod source" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922414 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 19:58:27.951739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922704 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.922718 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.925189 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.926827 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.926851 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.926994 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927006 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927012 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927017 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927024 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927030 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927564 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927570 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927577 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927582 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927589 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.927596 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.928340 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.928347 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.931066 2557 server.go:1268] "Started kubelet" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.931669 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.940873 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.940885 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.940874 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.931522 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.940952 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.941614 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.942219 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.943026 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.946530 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.946530 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.947955 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.947193 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-159.ec2.internal.18c28ee2fbd699a3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-159.ec2.internal,UID:ip-10-0-135-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-159.ec2.internal,},FirstTimestamp:2026-07-15 19:58:27.931036067 +0000 UTC m=+0.803029487,LastTimestamp:2026-07-15 19:58:27.931036067 +0000 UTC m=+0.803029487,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-159.ec2.internal,}" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.948517 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.948734 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.948747 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.948851 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.948857 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949606 2557 factory.go:55] Registering systemd factory Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949625 2557 factory.go:223] Registration of the systemd container factory successfully Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949831 2557 factory.go:153] Registering CRI-O factory Jul 15 19:58:27.952215 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949843 2557 factory.go:223] Registration of the crio container factory successfully Jul 15 19:58:27.953191 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949890 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 19:58:27.953191 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949907 2557 factory.go:103] Registering Raw factory Jul 15 19:58:27.953191 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.949916 2557 manager.go:1196] Started watching for new ooms in manager Jul 15 19:58:27.953191 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.950230 2557 manager.go:349] Starting recovery of all containers Jul 15 19:58:27.957567 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.957541 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2grn5" Jul 15 19:58:27.961529 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.961472 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 19:58:27.961678 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:27.961651 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 19:58:27.966663 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.966648 2557 manager.go:354] Recovery completed Jul 15 19:58:27.966925 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.966911 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2grn5" Jul 15 19:58:27.970650 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.970639 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:27.972436 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972423 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:27.972497 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972449 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:27.972497 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972458 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:27.972885 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972867 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 19:58:27.972885 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972883 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 19:58:27.973017 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972874 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:27.973017 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.972899 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:58:27.973487 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973474 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:27.973532 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973497 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:27.973532 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973506 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:27.973791 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973781 2557 policy_none.go:49] "None policy: Start" Jul 15 19:58:27.973825 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973796 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 19:58:27.973825 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.973806 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 19:58:27.974633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:27.974623 2557 policy_none.go:47] "Start" Jul 15 19:58:28.022308 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022159 2557 manager.go:337] "Starting Device Plugin manager" Jul 15 19:58:28.022399 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.022340 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 19:58:28.022399 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022352 2557 server.go:87] "Starting device plugin registration server" Jul 15 19:58:28.022570 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022560 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 19:58:28.022613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022573 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 19:58:28.022703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022682 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 19:58:28.022831 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022754 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 19:58:28.022831 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.022763 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 19:58:28.023193 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.023179 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 19:58:28.023288 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.023226 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.114490 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.114425 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 19:58:28.115640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.115541 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 19:58:28.115640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.115579 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 19:58:28.115640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.115601 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 19:58:28.115640 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.115638 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 19:58:28.117295 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.117276 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:28.122962 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.122948 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:28.123621 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.123607 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:28.123687 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.123632 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:28.123687 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.123643 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:28.123687 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.123663 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.133768 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.133753 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.133811 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.133772 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-159.ec2.internal\": node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.160846 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.160828 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.216611 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.216591 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal"] Jul 15 19:58:28.216687 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.216665 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:28.217422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.217405 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:28.217498 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.217432 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:28.217498 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.217441 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:28.218663 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.218653 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:28.218756 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.218744 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.218788 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.218770 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:28.219252 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219239 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:28.219309 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219242 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:28.219309 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219263 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:28.219309 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219274 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:28.219309 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219282 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:28.219309 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.219292 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:28.220625 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.220614 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.220664 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.220634 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:58:28.221191 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.221178 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:58:28.221267 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.221204 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:58:28.221267 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.221216 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:58:28.250977 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.250952 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19f77dbb92f973ab7c41f31009ab9ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-159.ec2.internal\" (UID: \"19f77dbb92f973ab7c41f31009ab9ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.251085 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.250984 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.251085 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.251008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.255544 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.255528 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-159.ec2.internal\" not found" node="ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.258714 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.258699 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-159.ec2.internal\" not found" node="ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.261648 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.261635 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.352066 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19f77dbb92f973ab7c41f31009ab9ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-159.ec2.internal\" (UID: \"19f77dbb92f973ab7c41f31009ab9ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.352166 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352084 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.352166 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.352166 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352147 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.352166 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352151 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19f77dbb92f973ab7c41f31009ab9ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-135-159.ec2.internal\" (UID: \"19f77dbb92f973ab7c41f31009ab9ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.352297 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.352147 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93e928ce34988777e7be9d4b28ce1ef4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal\" (UID: \"93e928ce34988777e7be9d4b28ce1ef4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.362164 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.362147 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.462259 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.462216 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.557578 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.557549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.560862 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.560840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.562676 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.562660 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.663403 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.663363 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.763791 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.763730 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-159.ec2.internal\" not found" Jul 15 19:58:28.801002 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.800984 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:28.848476 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.848458 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.858031 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.858017 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:58:28.859449 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.859439 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.860122 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.860110 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 19:58:28.860234 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.860218 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:58:28.860234 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.860220 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:58:28.860312 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.860221 2557 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a92aa028a1c2943eb80d0694bde8d3ee-774ccc6e87fb6a16.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.135.159:59568->54.167.36.161:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:28.879189 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.879169 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:28.923503 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.923485 2557 apiserver.go:52] "Watching apiserver" Jul 15 19:58:28.930237 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.930222 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 19:58:28.930991 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.930969 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal","openshift-dns/node-resolver-s2pzf","openshift-multus/multus-additional-cni-plugins-s4rg9","openshift-multus/multus-w9m6z","openshift-multus/network-metrics-daemon-dfwrk","openshift-network-diagnostics/network-check-target-6q45l","openshift-network-operator/iptables-alerter-nv8q8","kube-system/global-pull-secret-syncer-mdxkm","kube-system/konnectivity-agent-mfk9l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg","openshift-cluster-node-tuning-operator/tuned-lq89s","openshift-image-registry/node-ca-ql84l","openshift-ovn-kubernetes/ovnkube-node-77qz4"] Jul 15 19:58:28.932576 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.932562 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:28.933641 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.933619 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:28.934557 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.934543 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.935126 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.935112 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.935195 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.935164 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 19:58:28.935276 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.935207 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.935560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.935548 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n7csg\"" Jul 15 19:58:28.935736 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.935723 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.936322 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.936298 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rj857\"" Jul 15 19:58:28.936415 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.936345 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.936415 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.936383 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.936789 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.936777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:28.936845 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.936827 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:28.937200 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937185 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 19:58:28.937538 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937520 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.937538 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937529 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 19:58:28.937709 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937566 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.937709 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937520 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 19:58:28.937709 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937530 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9lbbx\"" Jul 15 19:58:28.937867 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937750 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:28.937867 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.937810 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 19:58:28.938159 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.938133 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c6d6v\"" Jul 15 19:58:28.939300 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.939185 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.940407 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.940351 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 19:58:28.940407 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.940350 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.940536 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.940445 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hmmcg\"" Jul 15 19:58:28.941060 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.941044 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 19:58:28.941677 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.941665 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.941780 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.941769 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 19:58:28.941895 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.941883 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7tmjg\"" Jul 15 19:58:28.942136 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.942123 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:28.942207 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.942162 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.942731 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.942719 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.943299 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.943283 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.943540 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.943524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:28.943613 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.943599 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:28.943645 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.943630 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zj8ql\"" Jul 15 19:58:28.944289 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.944276 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 19:58:28.944559 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.944543 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.944642 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.944602 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m6wcr\"" Jul 15 19:58:28.944642 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.944603 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.944730 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.944717 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:28.944787 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:28.944769 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:28.945880 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.945867 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.948140 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.948125 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 19:58:28.948244 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.948231 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 19:58:28.949237 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.949221 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 19:58:28.949502 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.949489 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 19:58:28.950120 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.950104 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 19:58:28.950120 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.950113 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 19:58:28.950251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.950163 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hq6xn\"" Jul 15 19:58:28.950251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.950210 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 19:58:28.950330 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.950268 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 19:58:28.956081 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956053 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-netns\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956101 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-systemd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956115 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-log-socket\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956127 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-bin\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-os-release\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956154 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956152 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-conf-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-socket-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-device-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956239 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-run\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-agent-certs\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956279 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-systemd-units\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956299 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-multus\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1366d093-4b76-41ac-8e20-f91deab4c3d8-iptables-alerter-script\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:28.956364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956361 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-etc-selinux\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-lib-modules\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-host\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956427 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-env-overrides\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovn-node-metrics-cert\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956470 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5e70159-25d3-43fd-a5da-aa768351e411-hosts-file\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956490 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-cni-binary-copy\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-registration-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-konnectivity-ca\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-ovn\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-node-log\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-netd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956613 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956607 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956653 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-kubelet\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956674 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-slash\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-system-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956713 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7bnw\" (UniqueName: \"kubernetes.io/projected/bb4fe0a3-63ed-4236-adb6-851e167f7320-kube-api-access-n7bnw\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a265d439-db82-4192-a282-098cffafd784-serviceca\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-hostroot\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956758 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e70159-25d3-43fd-a5da-aa768351e411-tmp-dir\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-var-lib-kubelet\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1366d093-4b76-41ac-8e20-f91deab4c3d8-host-slash\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc6k6\" (UniqueName: \"kubernetes.io/projected/1366d093-4b76-41ac-8e20-f91deab4c3d8-kube-api-access-sc6k6\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:28.956965 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956900 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-sys-fs\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-kubernetes\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpk96\" (UniqueName: \"kubernetes.io/projected/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-kube-api-access-vpk96\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.956998 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-kubelet\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-var-lib-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-config\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957080 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzw4\" (UniqueName: \"kubernetes.io/projected/1840d153-4cbf-447c-992f-2c01012eacbe-kube-api-access-8mzw4\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957101 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-os-release\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957122 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-systemd\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957142 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjws8\" (UniqueName: \"kubernetes.io/projected/664a51a4-4de8-4980-8379-41a1ca09e89b-kube-api-access-hjws8\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-socket-dir-parent\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-modprobe-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysconfig\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957229 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-k8s-cni-cncf-io\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957267 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmb2h\" (UniqueName: \"kubernetes.io/projected/be48c0fb-9fa9-44d9-984b-d03a39590b09-kube-api-access-pmb2h\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:28.957560 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957288 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-conf\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957307 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-tmp\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957334 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-script-lib\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957364 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957386 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-netns\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957399 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957421 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-tuned\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957446 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlfk6\" (UniqueName: \"kubernetes.io/projected/a265d439-db82-4192-a282-098cffafd784-kube-api-access-mlfk6\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957462 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957478 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-multus-certs\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957491 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-etc-kubernetes\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957505 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-kubelet-config\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957517 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-dbus\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957531 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6kqf\" (UniqueName: \"kubernetes.io/projected/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-kube-api-access-l6kqf\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8v4d\" (UniqueName: \"kubernetes.io/projected/d5e70159-25d3-43fd-a5da-aa768351e411-kube-api-access-q8v4d\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:28.958025 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a265d439-db82-4192-a282-098cffafd784-host\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957589 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-cnibin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957601 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-bin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-daemon-config\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957632 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-etc-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957657 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-sys\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:28.958466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.957675 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-cnibin\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:28.965547 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.965520 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:58:28.969056 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.969006 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:53:27 +0000 UTC" deadline="2028-03-03 11:06:35.137086214 +0000 UTC" Jul 15 19:58:28.969056 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.969048 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14319h8m6.168042686s" Jul 15 19:58:28.987835 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.987821 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7vpxz" Jul 15 19:58:28.996036 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:28.996023 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7vpxz" Jul 15 19:58:29.057892 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-etc-selinux\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-lib-modules\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057925 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-host\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-env-overrides\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057961 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovn-node-metrics-cert\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5e70159-25d3-43fd-a5da-aa768351e411-hosts-file\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.057998 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-cni-binary-copy\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058024 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-registration-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-konnectivity-ca\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058047 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-host\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058085 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-ovn\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058113 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058107 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-node-log\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-netd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058169 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-lib-modules\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-registration-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-kubelet\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-slash\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058265 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-kubelet\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058281 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-system-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7bnw\" (UniqueName: \"kubernetes.io/projected/bb4fe0a3-63ed-4236-adb6-851e167f7320-kube-api-access-n7bnw\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a265d439-db82-4192-a282-098cffafd784-serviceca\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058344 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-hostroot\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e70159-25d3-43fd-a5da-aa768351e411-tmp-dir\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058410 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058432 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.058633 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058453 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-var-lib-kubelet\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1366d093-4b76-41ac-8e20-f91deab4c3d8-host-slash\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058513 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc6k6\" (UniqueName: \"kubernetes.io/projected/1366d093-4b76-41ac-8e20-f91deab4c3d8-kube-api-access-sc6k6\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-etc-selinux\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-sys-fs\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058579 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-kubernetes\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058593 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-netd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058601 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpk96\" (UniqueName: \"kubernetes.io/projected/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-kube-api-access-vpk96\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-kubelet\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058640 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-node-log\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058644 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-var-lib-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-config\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058679 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzw4\" (UniqueName: \"kubernetes.io/projected/1840d153-4cbf-447c-992f-2c01012eacbe-kube-api-access-8mzw4\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-os-release\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.059366 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058718 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058737 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-systemd\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058765 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058767 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjws8\" (UniqueName: \"kubernetes.io/projected/664a51a4-4de8-4980-8379-41a1ca09e89b-kube-api-access-hjws8\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-socket-dir-parent\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058814 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-modprobe-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysconfig\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-k8s-cni-cncf-io\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmb2h\" (UniqueName: \"kubernetes.io/projected/be48c0fb-9fa9-44d9-984b-d03a39590b09-kube-api-access-pmb2h\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-konnectivity-ca\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-conf\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-tmp\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058955 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-script-lib\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059027 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1366d093-4b76-41ac-8e20-f91deab4c3d8-host-slash\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059087 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-netns\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-tuned\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlfk6\" (UniqueName: \"kubernetes.io/projected/a265d439-db82-4192-a282-098cffafd784-kube-api-access-mlfk6\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059179 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-multus-certs\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-etc-kubernetes\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-kubelet-config\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-dbus\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059284 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6kqf\" (UniqueName: \"kubernetes.io/projected/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-kube-api-access-l6kqf\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059313 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-os-release\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8v4d\" (UniqueName: \"kubernetes.io/projected/d5e70159-25d3-43fd-a5da-aa768351e411-kube-api-access-q8v4d\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a265d439-db82-4192-a282-098cffafd784-host\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-cnibin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059418 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-bin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-daemon-config\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059467 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-etc-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.060740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-sys\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-cnibin\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059541 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-netns\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059581 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-systemd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-log-socket\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-bin\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059648 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-os-release\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-conf-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.059689 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.059750 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:58:29.559734294 +0000 UTC m=+2.431727729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059771 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-socket-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059820 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-systemd\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059898 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-sys-fs\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-kubernetes\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060094 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-socket-dir-parent\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060149 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-ovn\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.061854 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060172 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-modprobe-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060188 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-kubelet\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060214 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysconfig\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060225 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-var-lib-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060249 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-k8s-cni-cncf-io\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060484 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-conf\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060713 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-netns\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060779 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.060877 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-env-overrides\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061152 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-etc-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061469 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a265d439-db82-4192-a282-098cffafd784-host\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061537 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-ovn-kubernetes\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061559 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-etc-kubernetes\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.058983 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-var-lib-kubelet\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061595 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-config\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.061995 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovnkube-script-lib\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.062651 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a265d439-db82-4192-a282-098cffafd784-serviceca\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062049 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/664a51a4-4de8-4980-8379-41a1ca09e89b-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062058 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-kubelet-config\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5e70159-25d3-43fd-a5da-aa768351e411-hosts-file\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062147 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-cnibin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062170 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/186e9fb1-120a-4998-8d07-1cf46a96d6eb-dbus\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062192 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-bin\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.059689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-socket-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062364 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-device-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-run\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-agent-certs\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062428 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-systemd-units\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062475 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-multus\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1366d093-4b76-41ac-8e20-f91deab4c3d8-iptables-alerter-script\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062536 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-cni-binary-copy\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062591 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-slash\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-system-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.063422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062770 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-systemd\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062899 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1366d093-4b76-41ac-8e20-f91deab4c3d8-iptables-alerter-script\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062914 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-hostroot\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.062963 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-sys\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063033 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/664a51a4-4de8-4980-8379-41a1ca09e89b-cnibin\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063144 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-sysctl-d\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063190 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-run-netns\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e70159-25d3-43fd-a5da-aa768351e411-tmp-dir\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-cni-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063452 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-systemd-units\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-run-multus-certs\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063513 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be48c0fb-9fa9-44d9-984b-d03a39590b09-device-dir\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.063564 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063595 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-host-var-lib-cni-multus\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063608 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-daemon-config\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.063618 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:58:29.563601479 +0000 UTC m=+2.435594899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063564 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-run-openvswitch\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.064222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-run\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063670 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-host-cni-bin\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063686 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-log-socket\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063687 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-os-release\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.063713 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4fe0a3-63ed-4236-adb6-851e167f7320-multus-conf-dir\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.064478 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-ovn-node-metrics-cert\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.065116 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.064949 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-etc-tuned\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.065384 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.065299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-tmp\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.065751 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.065726 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fece5b48-ad3e-4bc2-90f1-d489c87e29aa-agent-certs\") pod \"konnectivity-agent-mfk9l\" (UID: \"fece5b48-ad3e-4bc2-90f1-d489c87e29aa\") " pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:29.067263 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.067242 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzw4\" (UniqueName: \"kubernetes.io/projected/1840d153-4cbf-447c-992f-2c01012eacbe-kube-api-access-8mzw4\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:29.069259 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.069238 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8v4d\" (UniqueName: \"kubernetes.io/projected/d5e70159-25d3-43fd-a5da-aa768351e411-kube-api-access-q8v4d\") pod \"node-resolver-s2pzf\" (UID: \"d5e70159-25d3-43fd-a5da-aa768351e411\") " pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.069514 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.069495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc6k6\" (UniqueName: \"kubernetes.io/projected/1366d093-4b76-41ac-8e20-f91deab4c3d8-kube-api-access-sc6k6\") pod \"iptables-alerter-nv8q8\" (UID: \"1366d093-4b76-41ac-8e20-f91deab4c3d8\") " pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.069788 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.069772 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjws8\" (UniqueName: \"kubernetes.io/projected/664a51a4-4de8-4980-8379-41a1ca09e89b-kube-api-access-hjws8\") pod \"multus-additional-cni-plugins-s4rg9\" (UID: \"664a51a4-4de8-4980-8379-41a1ca09e89b\") " pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.072508 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.072478 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpk96\" (UniqueName: \"kubernetes.io/projected/e4adf01f-6ddc-4eae-9e85-8e0fffbfb295-kube-api-access-vpk96\") pod \"tuned-lq89s\" (UID: \"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295\") " pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.072842 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.072824 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlfk6\" (UniqueName: \"kubernetes.io/projected/a265d439-db82-4192-a282-098cffafd784-kube-api-access-mlfk6\") pod \"node-ca-ql84l\" (UID: \"a265d439-db82-4192-a282-098cffafd784\") " pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.072947 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.072928 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:29.073010 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.072955 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:29.073010 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.072967 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:29.073127 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.073038 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:29.573015688 +0000 UTC m=+2.445009112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:29.074398 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.074371 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7bnw\" (UniqueName: \"kubernetes.io/projected/bb4fe0a3-63ed-4236-adb6-851e167f7320-kube-api-access-n7bnw\") pod \"multus-w9m6z\" (UID: \"bb4fe0a3-63ed-4236-adb6-851e167f7320\") " pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.074622 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.074606 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6kqf\" (UniqueName: \"kubernetes.io/projected/1b6cbcd2-823e-470b-aab6-1d3e6f95723a-kube-api-access-l6kqf\") pod \"ovnkube-node-77qz4\" (UID: \"1b6cbcd2-823e-470b-aab6-1d3e6f95723a\") " pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.075014 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.074998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmb2h\" (UniqueName: \"kubernetes.io/projected/be48c0fb-9fa9-44d9-984b-d03a39590b09-kube-api-access-pmb2h\") pod \"aws-ebs-csi-driver-node-dvrhg\" (UID: \"be48c0fb-9fa9-44d9-984b-d03a39590b09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.092534 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.092516 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ql84l" Jul 15 19:58:29.098030 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.098016 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:29.142888 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.142862 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e928ce34988777e7be9d4b28ce1ef4.slice/crio-3686aa368265a8a95440f70faa16f65d89e34df3ab1fc50fea2d199f076c8f4b WatchSource:0}: Error finding container 3686aa368265a8a95440f70faa16f65d89e34df3ab1fc50fea2d199f076c8f4b: Status 404 returned error can't find the container with id 3686aa368265a8a95440f70faa16f65d89e34df3ab1fc50fea2d199f076c8f4b Jul 15 19:58:29.143256 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.143238 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f77dbb92f973ab7c41f31009ab9ff2.slice/crio-64c04af78cf5b53134e0d9cb796aae20f3c8c72375a94e799ca54ca9cd012181 WatchSource:0}: Error finding container 64c04af78cf5b53134e0d9cb796aae20f3c8c72375a94e799ca54ca9cd012181: Status 404 returned error can't find the container with id 64c04af78cf5b53134e0d9cb796aae20f3c8c72375a94e799ca54ca9cd012181 Jul 15 19:58:29.147969 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.147956 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:58:29.268408 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.268374 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nv8q8" Jul 15 19:58:29.276390 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.276368 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1366d093_4b76_41ac_8e20_f91deab4c3d8.slice/crio-b2de3746d93f0fe81af9b225f8202aa07322a6d1f90a77cf428764e9c8ec6fcd WatchSource:0}: Error finding container b2de3746d93f0fe81af9b225f8202aa07322a6d1f90a77cf428764e9c8ec6fcd: Status 404 returned error can't find the container with id b2de3746d93f0fe81af9b225f8202aa07322a6d1f90a77cf428764e9c8ec6fcd Jul 15 19:58:29.284279 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.284239 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2pzf" Jul 15 19:58:29.290798 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.290777 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e70159_25d3_43fd_a5da_aa768351e411.slice/crio-d5c535cf225f835c09a8162f79819700a78a265294a610640d16a143d464deaa WatchSource:0}: Error finding container d5c535cf225f835c09a8162f79819700a78a265294a610640d16a143d464deaa: Status 404 returned error can't find the container with id d5c535cf225f835c09a8162f79819700a78a265294a610640d16a143d464deaa Jul 15 19:58:29.298588 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.298571 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" Jul 15 19:58:29.298859 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.298845 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:29.305327 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.305308 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664a51a4_4de8_4980_8379_41a1ca09e89b.slice/crio-c17b77bf3e11a76ed08050d7375568200bedf220d657598d23d7cdd5a7834027 WatchSource:0}: Error finding container c17b77bf3e11a76ed08050d7375568200bedf220d657598d23d7cdd5a7834027: Status 404 returned error can't find the container with id c17b77bf3e11a76ed08050d7375568200bedf220d657598d23d7cdd5a7834027 Jul 15 19:58:29.312214 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.312200 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9m6z" Jul 15 19:58:29.318703 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.318682 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4fe0a3_63ed_4236_adb6_851e167f7320.slice/crio-4df3024b08b5fe1acfe31785b75fbdb773c8592b882d39d892cbd38a64c1d2ef WatchSource:0}: Error finding container 4df3024b08b5fe1acfe31785b75fbdb773c8592b882d39d892cbd38a64c1d2ef: Status 404 returned error can't find the container with id 4df3024b08b5fe1acfe31785b75fbdb773c8592b882d39d892cbd38a64c1d2ef Jul 15 19:58:29.329200 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.329180 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:29.337179 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.337159 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfece5b48_ad3e_4bc2_90f1_d489c87e29aa.slice/crio-c0c9fbfdee947a8e23cd977122349704f5e61489a3df020d2ce35af78b2732f0 WatchSource:0}: Error finding container c0c9fbfdee947a8e23cd977122349704f5e61489a3df020d2ce35af78b2732f0: Status 404 returned error can't find the container with id c0c9fbfdee947a8e23cd977122349704f5e61489a3df020d2ce35af78b2732f0 Jul 15 19:58:29.343982 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.343962 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" Jul 15 19:58:29.349883 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.349865 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe48c0fb_9fa9_44d9_984b_d03a39590b09.slice/crio-30457c165c39e2ca96a8b8bc93a2dcdfe59b740530fd5fd3897b46e60273bc1c WatchSource:0}: Error finding container 30457c165c39e2ca96a8b8bc93a2dcdfe59b740530fd5fd3897b46e60273bc1c: Status 404 returned error can't find the container with id 30457c165c39e2ca96a8b8bc93a2dcdfe59b740530fd5fd3897b46e60273bc1c Jul 15 19:58:29.363877 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.363856 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lq89s" Jul 15 19:58:29.369575 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.369558 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4adf01f_6ddc_4eae_9e85_8e0fffbfb295.slice/crio-2168b0f9e20d0bb11bdbd95190900c74a9cc33b31e7f8cf976ed78675aee47d9 WatchSource:0}: Error finding container 2168b0f9e20d0bb11bdbd95190900c74a9cc33b31e7f8cf976ed78675aee47d9: Status 404 returned error can't find the container with id 2168b0f9e20d0bb11bdbd95190900c74a9cc33b31e7f8cf976ed78675aee47d9 Jul 15 19:58:29.401169 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.401152 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6cbcd2_823e_470b_aab6_1d3e6f95723a.slice/crio-4a677b0119ee4a784db9e01ded94f97614bc759a8c485b9438cfdd2d079e600d WatchSource:0}: Error finding container 4a677b0119ee4a784db9e01ded94f97614bc759a8c485b9438cfdd2d079e600d: Status 404 returned error can't find the container with id 4a677b0119ee4a784db9e01ded94f97614bc759a8c485b9438cfdd2d079e600d Jul 15 19:58:29.432543 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:58:29.432521 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda265d439_db82_4192_a282_098cffafd784.slice/crio-b333f412b19474871ece0d0e77736e947a32ca78308b31bcb33060931299a2b9 WatchSource:0}: Error finding container b333f412b19474871ece0d0e77736e947a32ca78308b31bcb33060931299a2b9: Status 404 returned error can't find the container with id b333f412b19474871ece0d0e77736e947a32ca78308b31bcb33060931299a2b9 Jul 15 19:58:29.566405 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.566328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:29.566531 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.566499 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:29.566531 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.566517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:29.566615 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.566561 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:58:30.566542939 +0000 UTC m=+3.438536363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:29.566661 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.566629 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:29.566737 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.566721 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:58:30.566663756 +0000 UTC m=+3.438657168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:29.667701 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.667671 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:29.667871 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.667856 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:29.667939 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.667877 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:29.667939 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.667888 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:29.668048 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:29.667946 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:30.667929382 +0000 UTC m=+3.539922806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:29.739838 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.739811 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:29.997814 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.997703 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:53:28 +0000 UTC" deadline="2028-01-19 21:35:48.738735978 +0000 UTC" Jul 15 19:58:29.997814 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:29.997758 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13273h37m18.740981279s" Jul 15 19:58:30.117442 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.116859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:30.117442 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.116982 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:30.137926 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.137863 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"4a677b0119ee4a784db9e01ded94f97614bc759a8c485b9438cfdd2d079e600d"} Jul 15 19:58:30.153912 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.153832 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerStarted","Data":"c17b77bf3e11a76ed08050d7375568200bedf220d657598d23d7cdd5a7834027"} Jul 15 19:58:30.169288 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.169262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nv8q8" event={"ID":"1366d093-4b76-41ac-8e20-f91deab4c3d8","Type":"ContainerStarted","Data":"b2de3746d93f0fe81af9b225f8202aa07322a6d1f90a77cf428764e9c8ec6fcd"} Jul 15 19:58:30.189494 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.189465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" event={"ID":"93e928ce34988777e7be9d4b28ce1ef4","Type":"ContainerStarted","Data":"3686aa368265a8a95440f70faa16f65d89e34df3ab1fc50fea2d199f076c8f4b"} Jul 15 19:58:30.206328 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.206298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" event={"ID":"19f77dbb92f973ab7c41f31009ab9ff2","Type":"ContainerStarted","Data":"64c04af78cf5b53134e0d9cb796aae20f3c8c72375a94e799ca54ca9cd012181"} Jul 15 19:58:30.224530 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.224504 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ql84l" event={"ID":"a265d439-db82-4192-a282-098cffafd784","Type":"ContainerStarted","Data":"b333f412b19474871ece0d0e77736e947a32ca78308b31bcb33060931299a2b9"} Jul 15 19:58:30.228940 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.228914 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lq89s" event={"ID":"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295","Type":"ContainerStarted","Data":"2168b0f9e20d0bb11bdbd95190900c74a9cc33b31e7f8cf976ed78675aee47d9"} Jul 15 19:58:30.250876 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.250803 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" event={"ID":"be48c0fb-9fa9-44d9-984b-d03a39590b09","Type":"ContainerStarted","Data":"30457c165c39e2ca96a8b8bc93a2dcdfe59b740530fd5fd3897b46e60273bc1c"} Jul 15 19:58:30.285106 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.285046 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfk9l" event={"ID":"fece5b48-ad3e-4bc2-90f1-d489c87e29aa","Type":"ContainerStarted","Data":"c0c9fbfdee947a8e23cd977122349704f5e61489a3df020d2ce35af78b2732f0"} Jul 15 19:58:30.302744 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.302682 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9m6z" event={"ID":"bb4fe0a3-63ed-4236-adb6-851e167f7320","Type":"ContainerStarted","Data":"4df3024b08b5fe1acfe31785b75fbdb773c8592b882d39d892cbd38a64c1d2ef"} Jul 15 19:58:30.311804 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.311766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2pzf" event={"ID":"d5e70159-25d3-43fd-a5da-aa768351e411","Type":"ContainerStarted","Data":"d5c535cf225f835c09a8162f79819700a78a265294a610640d16a143d464deaa"} Jul 15 19:58:30.575717 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.575635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:30.575717 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.575681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:30.575911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.575832 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:30.575911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.575886 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:58:32.575869065 +0000 UTC m=+5.447862479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:30.576286 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.576270 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:30.576374 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.576318 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:58:32.576305175 +0000 UTC m=+5.448298585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:30.676882 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.676851 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:30.677042 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.677019 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:30.677124 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.677045 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:30.677124 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.677063 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:30.677207 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:30.677126 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:32.677110227 +0000 UTC m=+5.549103640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:30.998571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.998449 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:53:28 +0000 UTC" deadline="2028-02-07 06:18:57.040529046 +0000 UTC" Jul 15 19:58:30.998571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:30.998483 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13714h20m26.04205037s" Jul 15 19:58:31.117036 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:31.116441 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:31.117036 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:31.116568 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:31.117036 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:31.116910 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:31.117036 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:31.116988 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:31.568281 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:31.568233 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:58:32.118426 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:32.118398 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:32.118935 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.118507 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:32.593732 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:32.593693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:32.593902 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:32.593746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:32.593902 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.593877 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:32.593996 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.593932 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:58:36.593914963 +0000 UTC m=+9.465908373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:32.593996 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.593972 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:32.594130 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.594033 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:58:36.594017561 +0000 UTC m=+9.466010971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:32.694675 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:32.694638 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:32.694844 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.694789 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:32.694844 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.694803 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:32.694844 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.694812 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:32.695003 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:32.694856 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:36.6948439 +0000 UTC m=+9.566837308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:33.116587 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:33.116549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:33.116776 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:33.116720 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:33.116844 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:33.116549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:33.116974 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:33.116909 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:34.116367 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:34.116334 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:34.116829 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:34.116472 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:35.116678 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:35.116645 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:35.117121 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:35.116798 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:35.117121 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:35.116794 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:35.117121 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:35.116873 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:36.116018 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:36.115985 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:36.116205 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.116130 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:36.629544 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:36.629508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:36.630016 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:36.629555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:36.630016 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.629673 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:36.630016 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.629737 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.629720749 +0000 UTC m=+17.501714166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:36.630016 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.629674 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:36.630016 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.629803 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.629789645 +0000 UTC m=+17.501783056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:36.730884 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:36.730273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:36.730884 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.730467 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:36.730884 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.730485 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:36.730884 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.730498 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:36.730884 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:36.730554 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.730533015 +0000 UTC m=+17.602526447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:37.115920 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:37.115887 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:37.116111 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:37.115887 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:37.116111 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:37.116032 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:37.116111 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:37.116057 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:38.116998 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:38.116959 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:38.117484 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:38.117095 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:39.116251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:39.116214 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:39.116438 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:39.116209 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:39.116438 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:39.116350 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:39.116546 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:39.116451 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:40.116327 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:40.116294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:40.116772 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:40.116418 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:41.116427 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:41.116386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:41.116427 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:41.116422 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:41.116910 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:41.116528 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:41.116910 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:41.116642 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:42.116667 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:42.116626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:42.117172 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:42.116760 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:43.116640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:43.116602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:43.116826 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:43.116602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:43.116826 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:43.116730 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:43.117296 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:43.116832 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:44.116776 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:44.116739 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:44.116950 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.116866 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:44.685691 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:44.685662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:44.685691 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:44.685697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:44.685911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.685783 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:44.685911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.685796 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:44.685911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.685831 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:59:00.685817351 +0000 UTC m=+33.557810761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:58:44.685911 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.685843 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:59:00.685837192 +0000 UTC m=+33.557830603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:58:44.786847 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:44.786810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:44.787014 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.786974 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:58:44.787014 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.786990 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:58:44.787014 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.786999 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:44.787165 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:44.787056 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:00.787036353 +0000 UTC m=+33.659029806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:58:45.116520 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:45.116486 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:45.116520 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:45.116509 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:45.116754 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:45.116606 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:45.116754 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:45.116733 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:46.116228 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:46.116192 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:46.116787 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:46.116307 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:47.116526 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:47.116500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:47.116961 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:47.116600 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:47.116961 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:47.116673 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:47.116961 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:47.116775 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:48.116467 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:48.116439 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:48.116633 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:48.116533 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:49.116111 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.115929 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:49.116262 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.115992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:49.116262 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:49.116197 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:49.116340 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:49.116284 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:49.354210 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.354165 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" event={"ID":"19f77dbb92f973ab7c41f31009ab9ff2","Type":"ContainerStarted","Data":"28adff3ce2c0d3a4cfd17d08e17f6c00b4594904848bbfc7544a7e75b8524d46"} Jul 15 19:58:49.355740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.355716 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lq89s" event={"ID":"e4adf01f-6ddc-4eae-9e85-8e0fffbfb295","Type":"ContainerStarted","Data":"ef8d7e740f93439e0643a1b06fe67e618cc315343ce90ac413117faa4b377f09"} Jul 15 19:58:49.357364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.357338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9m6z" event={"ID":"bb4fe0a3-63ed-4236-adb6-851e167f7320","Type":"ContainerStarted","Data":"39458b7099209f31193a720ac97dc0b15769df964f3ec6982c8460ede6cba403"} Jul 15 19:58:49.368261 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.368238 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"f307cbd163fd4d4095543ed721bf86291ddceba51eac3fb1459379400423de56"} Jul 15 19:58:49.368372 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.368314 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"60bd592d0955e0031d201ba12fbdba05210e3a71b0080e523dcd9478c010fa45"} Jul 15 19:58:49.371130 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.371092 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-159.ec2.internal" podStartSLOduration=21.37106411 podStartE2EDuration="21.37106411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:49.37055462 +0000 UTC m=+22.242548054" watchObservedRunningTime="2026-07-15 19:58:49.37106411 +0000 UTC m=+22.243057542" Jul 15 19:58:49.390696 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.390658 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w9m6z" podStartSLOduration=3.0456884 podStartE2EDuration="22.390643605s" totalImagesPullingTime="19.344955205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:49.390317795 +0000 UTC m=+22.262311245" watchObservedRunningTime="2026-07-15 19:58:49.390643605 +0000 UTC m=+22.262637036" Jul 15 19:58:49.414495 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:49.412232 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lq89s" podStartSLOduration=3.151323892 podStartE2EDuration="22.412210897s" totalImagesPullingTime="19.260887005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:49.411836753 +0000 UTC m=+22.283830188" watchObservedRunningTime="2026-07-15 19:58:49.412210897 +0000 UTC m=+22.284204330" Jul 15 19:58:50.116345 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.116178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:50.116478 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:50.116423 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:50.371432 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.371159 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ql84l" event={"ID":"a265d439-db82-4192-a282-098cffafd784","Type":"ContainerStarted","Data":"9a1a98641b613dcc44511122b1caff2364514a81096055471d181bd2b068e750"} Jul 15 19:58:50.372842 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.372803 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" event={"ID":"be48c0fb-9fa9-44d9-984b-d03a39590b09","Type":"ContainerStarted","Data":"2106e66175982d5dcfd566c1b4fe431e6fd7c902da74e4f7ad185e297fbedfdc"} Jul 15 19:58:50.374337 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.374310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfk9l" event={"ID":"fece5b48-ad3e-4bc2-90f1-d489c87e29aa","Type":"ContainerStarted","Data":"405ea5191600c541c3387094301e3fe188ba0b1f494dc069dee1f5408783be93"} Jul 15 19:58:50.376049 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.376017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2pzf" event={"ID":"d5e70159-25d3-43fd-a5da-aa768351e411","Type":"ContainerStarted","Data":"0a71745edd3803668486f8b9431521a94c15a88aea83e5fb9225b08c85d77008"} Jul 15 19:58:50.379375 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379352 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 19:58:50.379763 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379729 2557 generic.go:356] "Generic (PLEG): container finished" podID="1b6cbcd2-823e-470b-aab6-1d3e6f95723a" containerID="f307cbd163fd4d4095543ed721bf86291ddceba51eac3fb1459379400423de56" exitCode=1 Jul 15 19:58:50.379866 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379792 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerDied","Data":"f307cbd163fd4d4095543ed721bf86291ddceba51eac3fb1459379400423de56"} Jul 15 19:58:50.379910 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379872 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"2648cef3cdf9ab18d5cf281f93a0e6429d0e10793b48599b4e798b274c7105a1"} Jul 15 19:58:50.379949 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379921 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"0db7adf69600e68298c7ee35270c6897a08397a039917b0d5a3c97d4a6a1fe8f"} Jul 15 19:58:50.380000 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.379970 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"1c422ed7dda381cc97b634a70d052d3f1853f7056f4416166a2f16ee956c92b4"} Jul 15 19:58:50.380055 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.380017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"f51ed646f4db10b41c8939a67faf1e4af4b0ad381639b9f08198f2ac321770da"} Jul 15 19:58:50.381912 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.381886 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="6a1e6dfa9344c32e996822a9d8bef43eb620cfba3355ccb781d1aa2cc73346a2" exitCode=0 Jul 15 19:58:50.382447 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.382423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"6a1e6dfa9344c32e996822a9d8bef43eb620cfba3355ccb781d1aa2cc73346a2"} Jul 15 19:58:50.385743 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.385579 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nv8q8" event={"ID":"1366d093-4b76-41ac-8e20-f91deab4c3d8","Type":"ContainerStarted","Data":"76e8e02dac56ee383be565b51150cd61f39792a29d9a01b07baa420fc899692d"} Jul 15 19:58:50.387350 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.387305 2557 generic.go:356] "Generic (PLEG): container finished" podID="93e928ce34988777e7be9d4b28ce1ef4" containerID="c5260199ef10b41b88046218117351506b1c40771036585ad2a53dcab4a931cb" exitCode=0 Jul 15 19:58:50.387439 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.387401 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" event={"ID":"93e928ce34988777e7be9d4b28ce1ef4","Type":"ContainerDied","Data":"c5260199ef10b41b88046218117351506b1c40771036585ad2a53dcab4a931cb"} Jul 15 19:58:50.387518 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.387494 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" Jul 15 19:58:50.392580 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.392545 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ql84l" podStartSLOduration=4.215018347 podStartE2EDuration="23.392533389s" totalImagesPullingTime="19.177515042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:50.392173439 +0000 UTC m=+23.264166871" watchObservedRunningTime="2026-07-15 19:58:50.392533389 +0000 UTC m=+23.264526821" Jul 15 19:58:50.411360 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.411345 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:58:50.412239 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.412219 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal"] Jul 15 19:58:50.415218 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.415183 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nv8q8" podStartSLOduration=4.064150311 podStartE2EDuration="23.415171226s" totalImagesPullingTime="19.351020915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:50.414816868 +0000 UTC m=+23.286810302" watchObservedRunningTime="2026-07-15 19:58:50.415171226 +0000 UTC m=+23.287164655" Jul 15 19:58:50.504591 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.504464 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s2pzf" podStartSLOduration=4.167751112 podStartE2EDuration="23.504443045s" totalImagesPullingTime="19.336691933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:50.504041209 +0000 UTC m=+23.376034643" watchObservedRunningTime="2026-07-15 19:58:50.504443045 +0000 UTC m=+23.376436478" Jul 15 19:58:50.504984 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.504948 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mfk9l" podStartSLOduration=4.214349923 podStartE2EDuration="23.504935919s" totalImagesPullingTime="19.290585996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:50.481772298 +0000 UTC m=+23.353765729" watchObservedRunningTime="2026-07-15 19:58:50.504935919 +0000 UTC m=+23.376929426" Jul 15 19:58:50.898862 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:50.898841 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 19:58:51.034513 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.034332 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:58:50.898858693Z","UUID":"c4fff2df-51e9-4190-a818-7f20333e69bd","Handler":null,"Name":"","Endpoint":""} Jul 15 19:58:51.036131 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.036109 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 19:58:51.036262 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.036174 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 19:58:51.116627 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.116598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:51.116855 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:51.116731 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:51.116855 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.116790 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:51.116979 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:51.116890 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:51.392658 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.392570 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" event={"ID":"be48c0fb-9fa9-44d9-984b-d03a39590b09","Type":"ContainerStarted","Data":"1fec9b829ea0459061bc10d9ad1dc44aff45491d57d4dbdafdef4118b8302e33"} Jul 15 19:58:51.396256 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.396230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" event={"ID":"93e928ce34988777e7be9d4b28ce1ef4","Type":"ContainerStarted","Data":"2f8b028243c0555676beeb28714edfa109ab1c4b4494b53d398fc678169679db"} Jul 15 19:58:51.414159 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:51.414105 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-159.ec2.internal" podStartSLOduration=1.414088353 podStartE2EDuration="1.414088353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:51.413524042 +0000 UTC m=+24.285517477" watchObservedRunningTime="2026-07-15 19:58:51.414088353 +0000 UTC m=+24.286081786" Jul 15 19:58:52.116211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:52.116190 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:52.116329 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:52.116305 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:52.400263 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:52.400181 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" event={"ID":"be48c0fb-9fa9-44d9-984b-d03a39590b09","Type":"ContainerStarted","Data":"2473f6cd26d380b08f30c58608bd49831b1099e2c5794ded14e86eef69a9f927"} Jul 15 19:58:52.403873 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:52.403846 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 19:58:52.404306 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:52.404277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"638929c7cc2c295f62614ff8b7d5b6626b2a58839912c138b827648eb165c176"} Jul 15 19:58:52.421229 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:52.421177 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dvrhg" podStartSLOduration=4.102994556 podStartE2EDuration="25.421158898s" totalImagesPullingTime="21.318164342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:52.420931012 +0000 UTC m=+25.292924446" watchObservedRunningTime="2026-07-15 19:58:52.421158898 +0000 UTC m=+25.293152329" Jul 15 19:58:53.116787 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.116747 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:53.116969 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.116747 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:53.116969 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:53.116873 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:53.116969 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:53.116915 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:53.124894 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.124866 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:53.125963 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.125944 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:53.406802 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.406709 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:53.407298 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:53.407279 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mfk9l" Jul 15 19:58:54.116271 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:54.116243 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:54.116455 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:54.116362 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:55.116311 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.116135 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:55.117003 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.116155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:55.117003 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:55.116395 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:55.117003 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:55.116451 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:55.413909 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.413848 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 19:58:55.414264 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.414242 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"e08b8bb56bf7cfdf424a30ab6e65ad59d651c06817fb33aa68df8c6fb4234719"} Jul 15 19:58:55.414485 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.414463 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:55.414671 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.414657 2557 scope.go:117] "RemoveContainer" containerID="f307cbd163fd4d4095543ed721bf86291ddceba51eac3fb1459379400423de56" Jul 15 19:58:55.416085 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.416051 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="ba062a7bbe6653072bb2ef1e889a5cc5e56aa929ad9e2673fe95c73613693d04" exitCode=0 Jul 15 19:58:55.416184 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.416099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"ba062a7bbe6653072bb2ef1e889a5cc5e56aa929ad9e2673fe95c73613693d04"} Jul 15 19:58:55.431313 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:55.431294 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:56.116261 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.116102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:56.116450 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:56.116341 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:56.422151 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.422129 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 19:58:56.422558 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.422534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" event={"ID":"1b6cbcd2-823e-470b-aab6-1d3e6f95723a","Type":"ContainerStarted","Data":"90253b33246ae01eb2af29cd9dcd86b620cf97b552eee5aa066f15ff04e0b06f"} Jul 15 19:58:56.422708 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.422693 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:56.422777 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.422757 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:56.424749 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.424727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerStarted","Data":"6a4a5a020eb515978db5ea6362c98a70bebe3bdf081830ac05511c089bdff8a2"} Jul 15 19:58:56.436820 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.436799 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:58:56.452409 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.452367 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" podStartSLOduration=8.723990866 podStartE2EDuration="28.452354093s" totalImagesPullingTime="19.728363227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:56.45043787 +0000 UTC m=+29.322431303" watchObservedRunningTime="2026-07-15 19:58:56.452354093 +0000 UTC m=+29.324347524" Jul 15 19:58:56.485981 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.485953 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mdxkm"] Jul 15 19:58:56.486119 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.486100 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:56.486225 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:56.486204 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:56.489256 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.489231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dfwrk"] Jul 15 19:58:56.489349 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.489324 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:56.489445 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:56.489419 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:56.489803 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.489781 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6q45l"] Jul 15 19:58:56.489878 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:56.489869 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:56.489968 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:56.489949 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:57.429175 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:57.429130 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="6a4a5a020eb515978db5ea6362c98a70bebe3bdf081830ac05511c089bdff8a2" exitCode=0 Jul 15 19:58:57.429530 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:57.429209 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"6a4a5a020eb515978db5ea6362c98a70bebe3bdf081830ac05511c089bdff8a2"} Jul 15 19:58:58.116728 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:58.116690 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:58:58.116891 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:58.116791 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:58:58.116891 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:58.116858 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:58:58.117004 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:58.116940 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:58:58.117004 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:58.116979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:58:58.117130 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:58:58.117044 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:58:59.435379 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:59.435343 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="a37d8a1cc2a8b54931df4b611519b9ef3e63011a7cc4e7dbdbbfdf1df0ef30bf" exitCode=0 Jul 15 19:58:59.435767 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:58:59.435394 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"a37d8a1cc2a8b54931df4b611519b9ef3e63011a7cc4e7dbdbbfdf1df0ef30bf"} Jul 15 19:59:00.115862 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.115832 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:00.116037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.115843 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:59:00.116037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.115948 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:00.116037 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.115945 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:59:00.116221 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.116044 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:59:00.116221 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.116133 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:59:00.712515 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.712490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:00.712515 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.712521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:59:00.712893 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.712606 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:59:00.712893 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.712620 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:59:00.712893 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.712653 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 19:59:32.712639079 +0000 UTC m=+65.584632488 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:59:00.712893 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.712674 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret podName:186e9fb1-120a-4998-8d07-1cf46a96d6eb nodeName:}" failed. No retries permitted until 2026-07-15 19:59:32.7126677 +0000 UTC m=+65.584661110 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret") pod "global-pull-secret-syncer-mdxkm" (UID: "186e9fb1-120a-4998-8d07-1cf46a96d6eb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:59:00.812881 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:00.812797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:00.813018 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.812934 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:59:00.813018 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.812954 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:59:00.813018 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.812968 2557 projected.go:196] Error preparing data for projected volume kube-api-access-zkwpx for pod openshift-network-diagnostics/network-check-target-6q45l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:59:00.813018 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:00.813015 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx podName:cfd5736e-1a82-4de5-80ff-0c3719e4cce4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:32.813002145 +0000 UTC m=+65.684995556 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zkwpx" (UniqueName: "kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx") pod "network-check-target-6q45l" (UID: "cfd5736e-1a82-4de5-80ff-0c3719e4cce4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:59:02.116813 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.116780 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:59:02.117282 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.116789 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:02.117282 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.116920 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:02.117282 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:02.116922 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfwrk" podUID="1840d153-4cbf-447c-992f-2c01012eacbe" Jul 15 19:59:02.117282 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:02.117004 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6q45l" podUID="cfd5736e-1a82-4de5-80ff-0c3719e4cce4" Jul 15 19:59:02.117282 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:02.117103 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mdxkm" podUID="186e9fb1-120a-4998-8d07-1cf46a96d6eb" Jul 15 19:59:02.931699 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.931418 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-159.ec2.internal" event="NodeReady" Jul 15 19:59:02.931855 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.931780 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 19:59:02.978723 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.978699 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 19:59:02.981127 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.981107 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:02.984456 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.984417 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 19:59:02.984456 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.984445 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 19:59:02.984739 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.984720 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqj2h\"" Jul 15 19:59:02.985139 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.985114 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 19:59:02.991030 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:02.991013 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 19:59:03.001643 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.001625 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w5zsr"] Jul 15 19:59:03.003399 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.003380 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.006763 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.006742 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 19:59:03.006914 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.006895 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wppjw\"" Jul 15 19:59:03.007317 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.007296 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 19:59:03.008421 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.008403 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 19:59:03.018600 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.018576 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5tcf7"] Jul 15 19:59:03.020302 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.020280 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.023038 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.023017 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 19:59:03.023665 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.023500 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tvpz5\"" Jul 15 19:59:03.023754 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.023738 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 19:59:03.024608 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.024550 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w5zsr"] Jul 15 19:59:03.025639 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.025612 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 19:59:03.034561 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.034538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5tcf7"] Jul 15 19:59:03.128823 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.128794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.128823 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.128828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.128935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.128997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129032 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ntjf\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-tmp-dir\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129096 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129128 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-config-volume\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129165 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfxft\" (UniqueName: \"kubernetes.io/projected/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-kube-api-access-vfxft\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129225 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/8a688de2-5672-448e-962b-6ea2707ab003-kube-api-access-l2kjb\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129252 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a688de2-5672-448e-962b-6ea2707ab003-cert\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129273 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.129377 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.129373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.230019 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.229944 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230019 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230001 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ntjf\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-tmp-dir\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230061 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-config-volume\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfxft\" (UniqueName: \"kubernetes.io/projected/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-kube-api-access-vfxft\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/8a688de2-5672-448e-962b-6ea2707ab003-kube-api-access-l2kjb\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a688de2-5672-448e-962b-6ea2707ab003-cert\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.230251 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.230247 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:03.230740 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.230270 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:03.230740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230284 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230311 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.230740 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.230342 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:03.730320802 +0000 UTC m=+36.602314234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:03.230740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.230634 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-tmp-dir\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.231066 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.231041 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.231187 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.231045 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-config-volume\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.231187 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.231147 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:03.231288 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.231209 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.231288 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.231227 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:03.731207258 +0000 UTC m=+36.603200681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:03.231396 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.231286 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.235178 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.235154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.235304 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.235177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a688de2-5672-448e-962b-6ea2707ab003-cert\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.235304 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.235156 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.239455 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.239435 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ntjf\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.239689 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.239668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.239883 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.239862 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfxft\" (UniqueName: \"kubernetes.io/projected/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-kube-api-access-vfxft\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.240007 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.239990 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/8a688de2-5672-448e-962b-6ea2707ab003-kube-api-access-l2kjb\") pod \"ingress-canary-5tcf7\" (UID: \"8a688de2-5672-448e-962b-6ea2707ab003\") " pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.334189 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.334160 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5tcf7" Jul 15 19:59:03.680769 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.680745 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5tcf7"] Jul 15 19:59:03.684334 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:03.684311 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a688de2_5672_448e_962b_6ea2707ab003.slice/crio-58315cde96909a2ade4e4977b2f56acf50089d58eb722c3beae1f5c97f0695f1 WatchSource:0}: Error finding container 58315cde96909a2ade4e4977b2f56acf50089d58eb722c3beae1f5c97f0695f1: Status 404 returned error can't find the container with id 58315cde96909a2ade4e4977b2f56acf50089d58eb722c3beae1f5c97f0695f1 Jul 15 19:59:03.733703 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.733677 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:03.733837 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:03.733721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:03.733898 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.733847 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:03.733898 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.733865 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:03.733898 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.733877 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:03.734055 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.733922 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:04.733906569 +0000 UTC m=+37.605899998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:03.734055 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:03.733937 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:04.73393061 +0000 UTC m=+37.605924020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:04.119653 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.119620 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:04.119812 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.119712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:04.119812 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.119794 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:59:04.122528 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.122509 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l6s2m\"" Jul 15 19:59:04.123777 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.123757 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:59:04.123863 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.123780 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:59:04.123863 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.123825 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:59:04.123960 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.123877 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:59:04.123960 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.123891 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kl6c8\"" Jul 15 19:59:04.448812 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.448719 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5tcf7" event={"ID":"8a688de2-5672-448e-962b-6ea2707ab003","Type":"ContainerStarted","Data":"58315cde96909a2ade4e4977b2f56acf50089d58eb722c3beae1f5c97f0695f1"} Jul 15 19:59:04.451909 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.451880 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="d42c0d0c1ba17fccf9d7040a7a85850e65b4bf179a03a1648b7d96bfaad144e1" exitCode=0 Jul 15 19:59:04.452030 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.451934 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"d42c0d0c1ba17fccf9d7040a7a85850e65b4bf179a03a1648b7d96bfaad144e1"} Jul 15 19:59:04.740200 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.740128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:04.740200 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:04.740174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:04.740384 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:04.740263 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:04.740384 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:04.740283 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:04.740384 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:04.740301 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:04.740384 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:04.740356 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:06.740339703 +0000 UTC m=+39.612333113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:04.740384 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:04.740373 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:06.740367152 +0000 UTC m=+39.612360562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:05.456585 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:05.456555 2557 generic.go:356] "Generic (PLEG): container finished" podID="664a51a4-4de8-4980-8379-41a1ca09e89b" containerID="2c95c7d831c60362ea44e1f76f2a7bbc7778f4eeaa14c4ea2466045964e137c5" exitCode=0 Jul 15 19:59:05.456919 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:05.456598 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerDied","Data":"2c95c7d831c60362ea44e1f76f2a7bbc7778f4eeaa14c4ea2466045964e137c5"} Jul 15 19:59:06.461861 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:06.461788 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" event={"ID":"664a51a4-4de8-4980-8379-41a1ca09e89b","Type":"ContainerStarted","Data":"eb2a20e4cf71664e2b78784e492f6e4899d1d47cc6cb6213394ce59112f2cc47"} Jul 15 19:59:06.504579 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:06.504529 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s4rg9" podStartSLOduration=9.661643134 podStartE2EDuration="39.504513752s" totalImagesPullingTime="29.842870618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:06.503281284 +0000 UTC m=+39.375274714" watchObservedRunningTime="2026-07-15 19:59:06.504513752 +0000 UTC m=+39.376507175" Jul 15 19:59:06.756414 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:06.756331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:06.756557 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:06.756443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:06.756557 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:06.756479 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:06.756557 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:06.756553 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:10.75653069 +0000 UTC m=+43.628524114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:06.756716 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:06.756563 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:06.756716 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:06.756578 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:06.756716 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:06.756626 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:10.756611512 +0000 UTC m=+43.628604936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:07.465780 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:07.465744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5tcf7" event={"ID":"8a688de2-5672-448e-962b-6ea2707ab003","Type":"ContainerStarted","Data":"2dce800ef056ea0bbb4569f197bb316f3d9ed45398f520a7c69b7244c5352144"} Jul 15 19:59:07.513327 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:07.513278 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5tcf7" podStartSLOduration=2.651101077 podStartE2EDuration="5.513263733s" totalImagesPullingTime="2.862162656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:07.512216014 +0000 UTC m=+40.384209445" watchObservedRunningTime="2026-07-15 19:59:07.513263733 +0000 UTC m=+40.385257209" Jul 15 19:59:10.786007 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:10.785968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:10.786097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:10.786154 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:10.786204 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:10.786216 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:10.786222 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:18.786203267 +0000 UTC m=+51.658196694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:10.786393 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:10.786247 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:18.786237462 +0000 UTC m=+51.658230872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:12.265538 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.265507 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf"] Jul 15 19:59:12.301363 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.301329 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rhkwb"] Jul 15 19:59:12.301516 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.301446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.304466 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.304238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nf6mj\"" Jul 15 19:59:12.305821 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.305781 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 19:59:12.306281 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.306240 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.307654 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.307635 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.323687 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.323665 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d485b46b6-r668j"] Jul 15 19:59:12.323796 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.323775 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.326581 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.326562 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.326877 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.326853 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bbngl\"" Jul 15 19:59:12.326974 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.326888 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.327026 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.326980 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 19:59:12.327740 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.327724 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 19:59:12.333448 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.333429 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 19:59:12.353535 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.353514 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf"] Jul 15 19:59:12.353649 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.353549 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rhkwb"] Jul 15 19:59:12.353649 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.353569 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d485b46b6-r668j"] Jul 15 19:59:12.353649 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.353617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.356362 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356338 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 19:59:12.356362 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356338 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wqp6p\"" Jul 15 19:59:12.356527 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356449 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.356527 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356504 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 19:59:12.356634 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356592 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.356790 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356773 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 19:59:12.356855 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.356771 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 19:59:12.371951 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.371927 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd"] Jul 15 19:59:12.391170 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.391142 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf"] Jul 15 19:59:12.391444 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.391423 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.394621 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.394605 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 19:59:12.394825 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.394811 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.395170 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.395153 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.395269 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.395255 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds4w6\" (UniqueName: \"kubernetes.io/projected/099450eb-3cf9-48be-b519-74d00f7f8690-kube-api-access-ds4w6\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.395434 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.395417 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lmcrv\"" Jul 15 19:59:12.395495 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.395423 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.396716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.396700 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 19:59:12.401242 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.401224 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd"] Jul 15 19:59:12.401321 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.401302 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.402037 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.402023 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-b72vx"] Jul 15 19:59:12.404152 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.404132 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 19:59:12.404497 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.404483 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 19:59:12.404681 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.404664 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.411967 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.411928 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ns9br\"" Jul 15 19:59:12.411967 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.411931 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.417523 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.417504 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp"] Jul 15 19:59:12.417640 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.417625 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.420618 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.420602 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 19:59:12.421218 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.421196 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 19:59:12.421307 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.421234 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.421307 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.421238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x9bds\"" Jul 15 19:59:12.421816 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.421778 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.427829 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.427803 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 19:59:12.431425 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.431402 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf"] Jul 15 19:59:12.431498 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.431441 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp"] Jul 15 19:59:12.431498 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.431459 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-b72vx"] Jul 15 19:59:12.431562 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.431506 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" Jul 15 19:59:12.434559 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.434541 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-d2gxs\"" Jul 15 19:59:12.477568 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.477544 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc"] Jul 15 19:59:12.493448 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.493424 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599"] Jul 15 19:59:12.494393 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.493557 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" Jul 15 19:59:12.495833 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.495809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.495944 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.495849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.495944 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.495878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-snapshots\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.495944 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.495918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p5w8\" (UniqueName: \"kubernetes.io/projected/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-kube-api-access-9p5w8\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.496125 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496013 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-tmp\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.496173 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5415a430-c08c-4476-b46c-b0a059e27d0b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.496219 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.496219 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds4w6\" (UniqueName: \"kubernetes.io/projected/099450eb-3cf9-48be-b519-74d00f7f8690-kube-api-access-ds4w6\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.496219 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496217 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.496346 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.496346 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496286 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.496346 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.496325 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:12.496486 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2b4d\" (UniqueName: \"kubernetes.io/projected/7be1a52a-e535-4911-8a11-94a4d0810d99-kube-api-access-b2b4d\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.496486 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496381 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-serving-cert\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.496486 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.496426 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:12.996407062 +0000 UTC m=+45.868400472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:12.496486 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.496668 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7q6s\" (UniqueName: \"kubernetes.io/projected/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-kube-api-access-n7q6s\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.496668 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496591 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-stats-auth\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.496668 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzm58\" (UniqueName: \"kubernetes.io/projected/5415a430-c08c-4476-b46c-b0a059e27d0b-kube-api-access-tzm58\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.496801 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496732 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.496801 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.496762 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-default-certificate\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.499782 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.499763 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.499882 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.499858 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sfd2f\"" Jul 15 19:59:12.499944 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.499926 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.509661 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.509643 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc"] Jul 15 19:59:12.509749 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.509690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599"] Jul 15 19:59:12.509787 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.509754 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.512939 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.512923 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-gpkrh\"" Jul 15 19:59:12.513130 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.513114 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 19:59:12.513248 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.513233 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:59:12.513315 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.513253 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 19:59:12.513619 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.513601 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 19:59:12.517467 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.517444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds4w6\" (UniqueName: \"kubernetes.io/projected/099450eb-3cf9-48be-b519-74d00f7f8690-kube-api-access-ds4w6\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:12.597241 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6sfv\" (UniqueName: \"kubernetes.io/projected/63c444be-8336-449f-9d09-51db84882997-kube-api-access-n6sfv\") pod \"network-check-source-869dcfcdcd-fxvmp\" (UID: \"63c444be-8336-449f-9d09-51db84882997\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" Jul 15 19:59:12.597241 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-trusted-ca\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.597420 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7q6s\" (UniqueName: \"kubernetes.io/projected/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-kube-api-access-n7q6s\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.597420 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-stats-auth\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.597420 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzm58\" (UniqueName: \"kubernetes.io/projected/5415a430-c08c-4476-b46c-b0a059e27d0b-kube-api-access-tzm58\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.597420 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597394 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xljl\" (UniqueName: \"kubernetes.io/projected/e4480f9f-c1e8-4483-848a-203f39ec0a92-kube-api-access-6xljl\") pod \"volume-data-source-validator-78495597f6-qcdzc\" (UID: \"e4480f9f-c1e8-4483-848a-203f39ec0a92\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597424 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-default-certificate\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597468 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-serving-cert\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597502 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.597571 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-snapshots\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p5w8\" (UniqueName: \"kubernetes.io/projected/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-kube-api-access-9p5w8\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-tmp\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5415a430-c08c-4476-b46c-b0a059e27d0b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsc92\" (UniqueName: \"kubernetes.io/projected/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-kube-api-access-zsc92\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.597808 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfee8f-3f1b-4686-915e-8971f0d072c5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bacfee8f-3f1b-4686-915e-8971f0d072c5-config\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597886 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nrpr\" (UniqueName: \"kubernetes.io/projected/bacfee8f-3f1b-4686-915e-8971f0d072c5-kube-api-access-6nrpr\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597923 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2b4d\" (UniqueName: \"kubernetes.io/projected/7be1a52a-e535-4911-8a11-94a4d0810d99-kube-api-access-b2b4d\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-serving-cert\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.597977 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598021 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-config\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.598211 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598176 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598264 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-snapshots\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598338 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-tmp\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.598362 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.598415 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:13.098397591 +0000 UTC m=+45.970391009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:12.598539 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.598470 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:13.098450316 +0000 UTC m=+45.970443752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:12.598861 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.598555 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:12.598861 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:12.598603 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:13.098590476 +0000 UTC m=+45.970583896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:12.598973 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.598886 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5415a430-c08c-4476-b46c-b0a059e27d0b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.599109 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.599091 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.600178 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.600154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-default-certificate\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.600281 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.600213 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-stats-auth\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.600498 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.600482 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.600714 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.600696 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-serving-cert\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.605720 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.605694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzm58\" (UniqueName: \"kubernetes.io/projected/5415a430-c08c-4476-b46c-b0a059e27d0b-kube-api-access-tzm58\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:12.607590 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.607566 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p5w8\" (UniqueName: \"kubernetes.io/projected/046bd0a4-42a9-4c26-a779-dd08c9aa8d4b-kube-api-access-9p5w8\") pod \"insights-operator-68d9dd94cf-rhkwb\" (UID: \"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b\") " pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.608008 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.607984 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7q6s\" (UniqueName: \"kubernetes.io/projected/e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8-kube-api-access-n7q6s\") pod \"kube-storage-version-migrator-operator-799cb7f958-hpnzf\" (UID: \"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.608364 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.608334 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2b4d\" (UniqueName: \"kubernetes.io/projected/7be1a52a-e535-4911-8a11-94a4d0810d99-kube-api-access-b2b4d\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:12.632716 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.632695 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" Jul 15 19:59:12.698835 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.698804 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsc92\" (UniqueName: \"kubernetes.io/projected/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-kube-api-access-zsc92\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.698989 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.698973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfee8f-3f1b-4686-915e-8971f0d072c5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.699055 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bacfee8f-3f1b-4686-915e-8971f0d072c5-config\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.699055 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nrpr\" (UniqueName: \"kubernetes.io/projected/bacfee8f-3f1b-4686-915e-8971f0d072c5-kube-api-access-6nrpr\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.699266 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-config\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.699332 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6sfv\" (UniqueName: \"kubernetes.io/projected/63c444be-8336-449f-9d09-51db84882997-kube-api-access-n6sfv\") pod \"network-check-source-869dcfcdcd-fxvmp\" (UID: \"63c444be-8336-449f-9d09-51db84882997\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" Jul 15 19:59:12.699332 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-trusted-ca\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.699435 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699356 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xljl\" (UniqueName: \"kubernetes.io/projected/e4480f9f-c1e8-4483-848a-203f39ec0a92-kube-api-access-6xljl\") pod \"volume-data-source-validator-78495597f6-qcdzc\" (UID: \"e4480f9f-c1e8-4483-848a-203f39ec0a92\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" Jul 15 19:59:12.699435 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-serving-cert\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.699662 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.699637 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bacfee8f-3f1b-4686-915e-8971f0d072c5-config\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.700106 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.700082 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-config\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.700403 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.700382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-trusted-ca\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.701303 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.701284 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfee8f-3f1b-4686-915e-8971f0d072c5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.701490 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.701473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-serving-cert\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.709164 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.709102 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nrpr\" (UniqueName: \"kubernetes.io/projected/bacfee8f-3f1b-4686-915e-8971f0d072c5-kube-api-access-6nrpr\") pod \"service-ca-operator-7695fc6c8c-pg599\" (UID: \"bacfee8f-3f1b-4686-915e-8971f0d072c5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.710190 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.709845 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6sfv\" (UniqueName: \"kubernetes.io/projected/63c444be-8336-449f-9d09-51db84882997-kube-api-access-n6sfv\") pod \"network-check-source-869dcfcdcd-fxvmp\" (UID: \"63c444be-8336-449f-9d09-51db84882997\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" Jul 15 19:59:12.710190 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.709954 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" Jul 15 19:59:12.710190 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.710041 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xljl\" (UniqueName: \"kubernetes.io/projected/e4480f9f-c1e8-4483-848a-203f39ec0a92-kube-api-access-6xljl\") pod \"volume-data-source-validator-78495597f6-qcdzc\" (UID: \"e4480f9f-c1e8-4483-848a-203f39ec0a92\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" Jul 15 19:59:12.711432 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.711411 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsc92\" (UniqueName: \"kubernetes.io/projected/7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4-kube-api-access-zsc92\") pod \"console-operator-7866497494-b72vx\" (UID: \"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4\") " pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.729995 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.729930 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:12.739350 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.739325 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" Jul 15 19:59:12.784146 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.784110 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rhkwb"] Jul 15 19:59:12.787446 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:12.787414 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046bd0a4_42a9_4c26_a779_dd08c9aa8d4b.slice/crio-92ca493ae43a46ed370951b046fb67cbccb32e0f5b7b53fddfd07865109b03fa WatchSource:0}: Error finding container 92ca493ae43a46ed370951b046fb67cbccb32e0f5b7b53fddfd07865109b03fa: Status 404 returned error can't find the container with id 92ca493ae43a46ed370951b046fb67cbccb32e0f5b7b53fddfd07865109b03fa Jul 15 19:59:12.804255 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.804225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" Jul 15 19:59:12.818722 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.818625 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" Jul 15 19:59:12.879688 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.879429 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf"] Jul 15 19:59:12.882656 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:12.882627 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7514dbf_3fe1_4b48_9c9b_0cda2b6a5ad8.slice/crio-c72786d7621cd581078dc71b8653042b943084d6e65fb69748aebd6885724cfa WatchSource:0}: Error finding container c72786d7621cd581078dc71b8653042b943084d6e65fb69748aebd6885724cfa: Status 404 returned error can't find the container with id c72786d7621cd581078dc71b8653042b943084d6e65fb69748aebd6885724cfa Jul 15 19:59:12.928649 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.928609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-b72vx"] Jul 15 19:59:12.933549 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:12.932816 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9df7fe_f6bd_4e80_8d9d_578a1bba8de4.slice/crio-82ec2908294b9e160c4102dfc6800f1e227e3ec0acd7a4c116c8e3b8af1cb0c0 WatchSource:0}: Error finding container 82ec2908294b9e160c4102dfc6800f1e227e3ec0acd7a4c116c8e3b8af1cb0c0: Status 404 returned error can't find the container with id 82ec2908294b9e160c4102dfc6800f1e227e3ec0acd7a4c116c8e3b8af1cb0c0 Jul 15 19:59:12.956521 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.956494 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp"] Jul 15 19:59:12.959302 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:12.959274 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c444be_8336_449f_9d09_51db84882997.slice/crio-94d14468557878cff712f921a3f6198b29da8ea2e51abe8566a224dd96c2af63 WatchSource:0}: Error finding container 94d14468557878cff712f921a3f6198b29da8ea2e51abe8566a224dd96c2af63: Status 404 returned error can't find the container with id 94d14468557878cff712f921a3f6198b29da8ea2e51abe8566a224dd96c2af63 Jul 15 19:59:12.986270 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:12.986219 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc"] Jul 15 19:59:12.992139 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:12.989167 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4480f9f_c1e8_4483_848a_203f39ec0a92.slice/crio-f2ed9f6d7823c5e054bb566c286d1f17e375953e5c71e04ab8c21afc0ada914f WatchSource:0}: Error finding container f2ed9f6d7823c5e054bb566c286d1f17e375953e5c71e04ab8c21afc0ada914f: Status 404 returned error can't find the container with id f2ed9f6d7823c5e054bb566c286d1f17e375953e5c71e04ab8c21afc0ada914f Jul 15 19:59:13.001829 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.001805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:13.001975 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.001957 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:13.002025 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.002016 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:14.002001087 +0000 UTC m=+46.873994497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:13.017195 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.017167 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599"] Jul 15 19:59:13.019985 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:13.019961 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacfee8f_3f1b_4686_915e_8971f0d072c5.slice/crio-28cce87b1aecde30fcf4856d49acbc00f549b59a0712d74e538dd4460122363f WatchSource:0}: Error finding container 28cce87b1aecde30fcf4856d49acbc00f549b59a0712d74e538dd4460122363f: Status 404 returned error can't find the container with id 28cce87b1aecde30fcf4856d49acbc00f549b59a0712d74e538dd4460122363f Jul 15 19:59:13.102477 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.102435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:13.102673 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.102577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:13.102673 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.102604 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:13.102673 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.102626 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:14.10260695 +0000 UTC m=+46.974600360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:13.102833 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.102695 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:13.102833 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.102721 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:13.102833 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.102753 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:14.102737729 +0000 UTC m=+46.974731141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:13.102833 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:13.102772 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:14.102762669 +0000 UTC m=+46.974756094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:13.480405 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.480364 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" event={"ID":"bacfee8f-3f1b-4686-915e-8971f0d072c5","Type":"ContainerStarted","Data":"28cce87b1aecde30fcf4856d49acbc00f549b59a0712d74e538dd4460122363f"} Jul 15 19:59:13.481398 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.481376 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" event={"ID":"e4480f9f-c1e8-4483-848a-203f39ec0a92","Type":"ContainerStarted","Data":"f2ed9f6d7823c5e054bb566c286d1f17e375953e5c71e04ab8c21afc0ada914f"} Jul 15 19:59:13.482359 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.482334 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" event={"ID":"63c444be-8336-449f-9d09-51db84882997","Type":"ContainerStarted","Data":"94d14468557878cff712f921a3f6198b29da8ea2e51abe8566a224dd96c2af63"} Jul 15 19:59:13.483458 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.483436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-b72vx" event={"ID":"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4","Type":"ContainerStarted","Data":"82ec2908294b9e160c4102dfc6800f1e227e3ec0acd7a4c116c8e3b8af1cb0c0"} Jul 15 19:59:13.484418 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.484397 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" event={"ID":"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8","Type":"ContainerStarted","Data":"c72786d7621cd581078dc71b8653042b943084d6e65fb69748aebd6885724cfa"} Jul 15 19:59:13.485356 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:13.485337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" event={"ID":"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b","Type":"ContainerStarted","Data":"92ca493ae43a46ed370951b046fb67cbccb32e0f5b7b53fddfd07865109b03fa"} Jul 15 19:59:14.011610 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:14.011568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:14.011821 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.011802 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:14.011894 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.011874 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.011853927 +0000 UTC m=+48.883847340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:14.113607 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:14.112828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:14.113607 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:14.112881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:14.113607 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:14.112936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:14.113607 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.113173 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.113140956 +0000 UTC m=+48.985134388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:14.113607 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.113580 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:14.114004 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.113635 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.113620152 +0000 UTC m=+48.985613574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:14.114004 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.113580 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:14.114004 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:14.113674 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.113665925 +0000 UTC m=+48.985659340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:16.032985 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:16.032954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:16.033410 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.033143 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:16.033410 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.033230 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:20.033209071 +0000 UTC m=+52.905202486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:16.133761 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:16.133724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:16.133761 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:16.133764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:16.133936 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:16.133799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:16.133936 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.133848 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:16.133936 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.133909 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:16.134088 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.133915 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:20.133895449 +0000 UTC m=+53.005888878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:16.134088 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.133994 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:20.133968878 +0000 UTC m=+53.005962300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:16.134088 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:16.134021 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:20.134010142 +0000 UTC m=+53.006003559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:18.853532 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:18.853485 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:18.853595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:18.853645 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:18.853691 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:18.853706 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884f584f6-xx7cb: secret "image-registry-tls" not found Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:18.853751 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls podName:fbcfe517-551b-4abd-9dd0-b1fab5ddb979 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:34.853726227 +0000 UTC m=+67.725719680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls") pod "dns-default-w5zsr" (UID: "fbcfe517-551b-4abd-9dd0-b1fab5ddb979") : secret "dns-default-metrics-tls" not found Jul 15 19:59:18.853957 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:18.853773 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls podName:5c41ff42-68be-46ed-9ad5-685fe9e7e8b5 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:34.853763631 +0000 UTC m=+67.725757043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls") pod "image-registry-6884f584f6-xx7cb" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5") : secret "image-registry-tls" not found Jul 15 19:59:19.507653 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.507624 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" event={"ID":"e4480f9f-c1e8-4483-848a-203f39ec0a92","Type":"ContainerStarted","Data":"29a77d99d988aecf9a47f50197c982a98d7d82b4cadeff03ea211beff0b788dc"} Jul 15 19:59:19.510159 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.510005 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" event={"ID":"63c444be-8336-449f-9d09-51db84882997","Type":"ContainerStarted","Data":"9d40fdf83f88fdcd27e38ba1b63b335d89c9260f5959ad04b133eb69407b5f45"} Jul 15 19:59:19.512922 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.512332 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" event={"ID":"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8","Type":"ContainerStarted","Data":"dbd672ee9edcd9a79249247306b4252a91f53ae1a3edf74cefa8e62f27b932c7"} Jul 15 19:59:19.515423 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.515403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" event={"ID":"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b","Type":"ContainerStarted","Data":"21e04a1b98325866c785a3f22451bcec7fa4b1fc865e736c4e5542c05e05aca4"} Jul 15 19:59:19.528452 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.528175 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-fxvmp" podStartSLOduration=1.179797976 podStartE2EDuration="7.528156959s" totalImagesPullingTime="6.348358983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:19.526157292 +0000 UTC m=+52.398150727" watchObservedRunningTime="2026-07-15 19:59:19.528156959 +0000 UTC m=+52.400150388" Jul 15 19:59:19.570731 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.570676 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" podStartSLOduration=1.050682514 podStartE2EDuration="7.570655632s" totalImagesPullingTime="6.519973118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:19.569735364 +0000 UTC m=+52.441728796" watchObservedRunningTime="2026-07-15 19:59:19.570655632 +0000 UTC m=+52.442649065" Jul 15 19:59:19.571234 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:19.571197 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" podStartSLOduration=1.175328328 podStartE2EDuration="7.571185424s" totalImagesPullingTime="6.395857096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:19.548615129 +0000 UTC m=+52.420608562" watchObservedRunningTime="2026-07-15 19:59:19.571185424 +0000 UTC m=+52.443178856" Jul 15 19:59:20.064685 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.064654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:20.065124 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.064829 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:20.065124 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.064910 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.064888244 +0000 UTC m=+60.936881671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:20.165709 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.165674 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:20.165864 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.165821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:20.165864 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.165849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:20.165983 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.165957 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:20.166030 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.166014 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.165995839 +0000 UTC m=+61.037989255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:20.166255 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.166201 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:20.166255 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.166243 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.166223818 +0000 UTC m=+61.038217231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:20.166410 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:20.166269 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.166258597 +0000 UTC m=+61.038252013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:20.521032 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.520955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" event={"ID":"bacfee8f-3f1b-4686-915e-8971f0d072c5","Type":"ContainerStarted","Data":"e05972985328b99967b552083d28c3a57d6e07c38bc91f52b0e77a819a6d8a25"} Jul 15 19:59:20.522736 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.522712 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/0.log" Jul 15 19:59:20.522848 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.522755 2557 generic.go:356] "Generic (PLEG): container finished" podID="7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4" containerID="f5b8cbf7de5111c080f90ff4743b256f08a26c6f4e99ad2bcf3157ad64b10989" exitCode=255 Jul 15 19:59:20.522848 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.522823 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-b72vx" event={"ID":"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4","Type":"ContainerDied","Data":"f5b8cbf7de5111c080f90ff4743b256f08a26c6f4e99ad2bcf3157ad64b10989"} Jul 15 19:59:20.523014 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.523001 2557 scope.go:117] "RemoveContainer" containerID="f5b8cbf7de5111c080f90ff4743b256f08a26c6f4e99ad2bcf3157ad64b10989" Jul 15 19:59:20.550045 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.549998 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" podStartSLOduration=2.239542168 podStartE2EDuration="8.549982838s" totalImagesPullingTime="6.31044067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:20.54992011 +0000 UTC m=+53.421913541" watchObservedRunningTime="2026-07-15 19:59:20.549982838 +0000 UTC m=+53.421976271" Jul 15 19:59:20.620300 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.620240 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-qcdzc" podStartSLOduration=2.303659086 podStartE2EDuration="8.620220654s" totalImagesPullingTime="6.316561568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:20.619652257 +0000 UTC m=+53.491645691" watchObservedRunningTime="2026-07-15 19:59:20.620220654 +0000 UTC m=+53.492214089" Jul 15 19:59:20.962598 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.962566 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7sfd9"] Jul 15 19:59:20.978417 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.978385 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:20.979636 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.979613 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7sfd9"] Jul 15 19:59:20.981369 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.981349 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v5glv\"" Jul 15 19:59:20.981599 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.981582 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 19:59:20.981900 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:20.981887 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 19:59:21.076354 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.076328 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b78b0486-62b8-4cf0-97d8-fd8682d6604a-crio-socket\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.076648 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.076376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.076648 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.076466 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b78b0486-62b8-4cf0-97d8-fd8682d6604a-data-volume\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.076648 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.076509 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm29w\" (UniqueName: \"kubernetes.io/projected/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-api-access-cm29w\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.076648 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.076596 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.177926 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.177898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178054 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.177946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b78b0486-62b8-4cf0-97d8-fd8682d6604a-crio-socket\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178054 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.178013 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:59:21.178147 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.178097 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:21.678063686 +0000 UTC m=+54.550057115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : configmap "kube-rbac-proxy" not found Jul 15 19:59:21.178147 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.178120 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b78b0486-62b8-4cf0-97d8-fd8682d6604a-crio-socket\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178147 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.178122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178242 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.178202 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:21.178242 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.178205 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b78b0486-62b8-4cf0-97d8-fd8682d6604a-data-volume\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178242 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.178240 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:21.678229065 +0000 UTC m=+54.550222475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:21.178330 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.178280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm29w\" (UniqueName: \"kubernetes.io/projected/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-api-access-cm29w\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.178500 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.178485 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b78b0486-62b8-4cf0-97d8-fd8682d6604a-data-volume\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.200718 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.200687 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm29w\" (UniqueName: \"kubernetes.io/projected/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-api-access-cm29w\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.529905 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.529877 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 19:59:21.530328 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.530314 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/0.log" Jul 15 19:59:21.530422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.530347 2557 generic.go:356] "Generic (PLEG): container finished" podID="7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4" containerID="c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4" exitCode=255 Jul 15 19:59:21.530422 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.530379 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-b72vx" event={"ID":"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4","Type":"ContainerDied","Data":"c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4"} Jul 15 19:59:21.530516 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.530446 2557 scope.go:117] "RemoveContainer" containerID="f5b8cbf7de5111c080f90ff4743b256f08a26c6f4e99ad2bcf3157ad64b10989" Jul 15 19:59:21.530682 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.530656 2557 scope.go:117] "RemoveContainer" containerID="c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4" Jul 15 19:59:21.530875 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.530840 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-b72vx_openshift-console-operator(7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4)\"" pod="openshift-console-operator/console-operator-7866497494-b72vx" podUID="7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4" Jul 15 19:59:21.682814 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.682783 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.682922 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:21.682866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:21.682922 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.682872 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:59:21.686323 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.683332 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.683306081 +0000 UTC m=+55.555299491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : configmap "kube-rbac-proxy" not found Jul 15 19:59:21.686323 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.683348 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:21.686323 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:21.683419 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.683399708 +0000 UTC m=+55.555393121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:22.534668 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.534641 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 19:59:22.535005 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.534951 2557 scope.go:117] "RemoveContainer" containerID="c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4" Jul 15 19:59:22.535148 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:22.535129 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-b72vx_openshift-console-operator(7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4)\"" pod="openshift-console-operator/console-operator-7866497494-b72vx" podUID="7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4" Jul 15 19:59:22.693304 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.693269 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:22.693519 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.693351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:22.693519 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:22.693377 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:59:22.693519 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:22.693431 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:24.693416628 +0000 UTC m=+57.565410038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : configmap "kube-rbac-proxy" not found Jul 15 19:59:22.693519 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:22.693463 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:22.693519 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:22.693504 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:24.693490839 +0000 UTC m=+57.565484249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:22.730401 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.730378 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:22.730513 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.730425 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:22.928774 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:22.928753 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2pzf_d5e70159-25d3-43fd-a5da-aa768351e411/dns-node-resolver/0.log" Jul 15 19:59:23.538194 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:23.538168 2557 scope.go:117] "RemoveContainer" containerID="c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4" Jul 15 19:59:23.538541 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:23.538311 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-b72vx_openshift-console-operator(7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4)\"" pod="openshift-console-operator/console-operator-7866497494-b72vx" podUID="7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4" Jul 15 19:59:24.126671 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:24.126645 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ql84l_a265d439-db82-4192-a282-098cffafd784/node-ca/0.log" Jul 15 19:59:24.526013 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:24.525917 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5tcf7_8a688de2-5672-448e-962b-6ea2707ab003/serve-healthcheck-canary/0.log" Jul 15 19:59:24.709735 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:24.709702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:24.710123 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:24.709766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:24.710123 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:24.709824 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:59:24.710123 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:24.709874 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:24.710123 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:24.709887 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.709872247 +0000 UTC m=+61.581865658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : configmap "kube-rbac-proxy" not found Jul 15 19:59:24.710123 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:24.709919 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.709904179 +0000 UTC m=+61.581897590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:28.136420 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.136386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:28.136741 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.136526 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:59:28.136741 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.136580 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls podName:099450eb-3cf9-48be-b519-74d00f7f8690 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.136565719 +0000 UTC m=+77.008559132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls") pod "cluster-samples-operator-65db858894-nlprf" (UID: "099450eb-3cf9-48be-b519-74d00f7f8690") : secret "samples-operator-tls" not found Jul 15 19:59:28.236790 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.236756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:28.236933 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.236893 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:28.236933 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.236913 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:28.237009 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.236933 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.236915331 +0000 UTC m=+77.108908743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : configmap references non-existent config key: service-ca.crt Jul 15 19:59:28.237059 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.237033 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:28.237059 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.237041 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:59:28.237148 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.237096 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs podName:7be1a52a-e535-4911-8a11-94a4d0810d99 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.237083464 +0000 UTC m=+77.109076887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs") pod "router-default-6d485b46b6-r668j" (UID: "7be1a52a-e535-4911-8a11-94a4d0810d99") : secret "router-metrics-certs-default" not found Jul 15 19:59:28.237148 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.237111 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls podName:5415a430-c08c-4476-b46c-b0a059e27d0b nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.237105254 +0000 UTC m=+77.109098663 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-v7kjd" (UID: "5415a430-c08c-4476-b46c-b0a059e27d0b") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:59:28.449255 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.449192 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77qz4" Jul 15 19:59:28.742325 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.742234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:28.742485 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.742375 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:28.742485 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:28.742384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:28.742485 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.742442 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:36.742425504 +0000 UTC m=+69.614418915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:28.742485 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.742470 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:59:28.742660 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:28.742540 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:36.742522584 +0000 UTC m=+69.614516008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : configmap "kube-rbac-proxy" not found Jul 15 19:59:32.774474 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.774435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 19:59:32.774880 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.774568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:32.779014 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.778994 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:59:32.779189 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.779173 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:59:32.785413 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:32.785398 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:59:32.785469 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:32.785449 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs podName:1840d153-4cbf-447c-992f-2c01012eacbe nodeName:}" failed. No retries permitted until 2026-07-15 20:00:36.785434644 +0000 UTC m=+129.657428055 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs") pod "network-metrics-daemon-dfwrk" (UID: "1840d153-4cbf-447c-992f-2c01012eacbe") : secret "metrics-daemon-secret" not found Jul 15 19:59:32.787382 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.787353 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/186e9fb1-120a-4998-8d07-1cf46a96d6eb-original-pull-secret\") pod \"global-pull-secret-syncer-mdxkm\" (UID: \"186e9fb1-120a-4998-8d07-1cf46a96d6eb\") " pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:32.875619 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.875585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:32.878128 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.878108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkwpx\" (UniqueName: \"kubernetes.io/projected/cfd5736e-1a82-4de5-80ff-0c3719e4cce4-kube-api-access-zkwpx\") pod \"network-check-target-6q45l\" (UID: \"cfd5736e-1a82-4de5-80ff-0c3719e4cce4\") " pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:32.932771 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.932748 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l6s2m\"" Jul 15 19:59:32.934857 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.934842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mdxkm" Jul 15 19:59:32.939920 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:32.939905 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:33.079155 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.079129 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mdxkm"] Jul 15 19:59:33.082199 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:33.082176 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186e9fb1_120a_4998_8d07_1cf46a96d6eb.slice/crio-8efeae515d3f79c1a094f7b74945ca89a7836933a7525512887840bdd765fb09 WatchSource:0}: Error finding container 8efeae515d3f79c1a094f7b74945ca89a7836933a7525512887840bdd765fb09: Status 404 returned error can't find the container with id 8efeae515d3f79c1a094f7b74945ca89a7836933a7525512887840bdd765fb09 Jul 15 19:59:33.101335 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.101254 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6q45l"] Jul 15 19:59:33.103435 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:33.103418 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd5736e_1a82_4de5_80ff_0c3719e4cce4.slice/crio-a3031674e08c3a34521dfd0ae6b0f7c5202975a7d0cd0cd8f7ba10397c54caaa WatchSource:0}: Error finding container a3031674e08c3a34521dfd0ae6b0f7c5202975a7d0cd0cd8f7ba10397c54caaa: Status 404 returned error can't find the container with id a3031674e08c3a34521dfd0ae6b0f7c5202975a7d0cd0cd8f7ba10397c54caaa Jul 15 19:59:33.568936 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.568897 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6q45l" event={"ID":"cfd5736e-1a82-4de5-80ff-0c3719e4cce4","Type":"ContainerStarted","Data":"87e5eaf8f03d189d1ea8bedb528ab92ddc4a6898dee7eeafb1294f47c524abb5"} Jul 15 19:59:33.569208 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.569187 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 19:59:33.569365 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.569326 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6q45l" event={"ID":"cfd5736e-1a82-4de5-80ff-0c3719e4cce4","Type":"ContainerStarted","Data":"a3031674e08c3a34521dfd0ae6b0f7c5202975a7d0cd0cd8f7ba10397c54caaa"} Jul 15 19:59:33.570200 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.570167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mdxkm" event={"ID":"186e9fb1-120a-4998-8d07-1cf46a96d6eb","Type":"ContainerStarted","Data":"8efeae515d3f79c1a094f7b74945ca89a7836933a7525512887840bdd765fb09"} Jul 15 19:59:33.592261 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:33.592215 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6q45l" podStartSLOduration=66.592197893 podStartE2EDuration="1m6.592197893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:33.591139155 +0000 UTC m=+66.463132588" watchObservedRunningTime="2026-07-15 19:59:33.592197893 +0000 UTC m=+66.464191335" Jul 15 19:59:34.895222 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:34.895168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:34.895679 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:34.895277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:34.898218 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:34.898194 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fbcfe517-551b-4abd-9dd0-b1fab5ddb979-metrics-tls\") pod \"dns-default-w5zsr\" (UID: \"fbcfe517-551b-4abd-9dd0-b1fab5ddb979\") " pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:34.898495 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:34.898473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"image-registry-6884f584f6-xx7cb\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:35.095405 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.095381 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqj2h\"" Jul 15 19:59:35.102905 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.102876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:35.115744 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.115721 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wppjw\"" Jul 15 19:59:35.123268 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.123254 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:35.282928 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.282892 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 19:59:35.285851 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:35.285816 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c41ff42_68be_46ed_9ad5_685fe9e7e8b5.slice/crio-d686af85598b41c722c5c5001bd6feee23894a4bbb16b9d76f0e19e2f3933bbb WatchSource:0}: Error finding container d686af85598b41c722c5c5001bd6feee23894a4bbb16b9d76f0e19e2f3933bbb: Status 404 returned error can't find the container with id d686af85598b41c722c5c5001bd6feee23894a4bbb16b9d76f0e19e2f3933bbb Jul 15 19:59:35.301712 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.301692 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w5zsr"] Jul 15 19:59:35.304219 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:35.304193 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcfe517_551b_4abd_9dd0_b1fab5ddb979.slice/crio-978969ff6e82f5d27c24cbd352ff181b380cc1273746fa30735eae94813b4e35 WatchSource:0}: Error finding container 978969ff6e82f5d27c24cbd352ff181b380cc1273746fa30735eae94813b4e35: Status 404 returned error can't find the container with id 978969ff6e82f5d27c24cbd352ff181b380cc1273746fa30735eae94813b4e35 Jul 15 19:59:35.581664 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.581624 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" event={"ID":"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5","Type":"ContainerStarted","Data":"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77"} Jul 15 19:59:35.582126 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.581694 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:35.582126 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.581713 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" event={"ID":"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5","Type":"ContainerStarted","Data":"d686af85598b41c722c5c5001bd6feee23894a4bbb16b9d76f0e19e2f3933bbb"} Jul 15 19:59:35.583114 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.583066 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5zsr" event={"ID":"fbcfe517-551b-4abd-9dd0-b1fab5ddb979","Type":"ContainerStarted","Data":"978969ff6e82f5d27c24cbd352ff181b380cc1273746fa30735eae94813b4e35"} Jul 15 19:59:35.606676 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:35.606628 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" podStartSLOduration=67.606613263 podStartE2EDuration="1m7.606613263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:35.605195044 +0000 UTC m=+68.477188473" watchObservedRunningTime="2026-07-15 19:59:35.606613263 +0000 UTC m=+68.478606673" Jul 15 19:59:36.813171 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:36.813135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:36.813609 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:36.813236 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:36.813609 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:36.813401 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:59:36.813609 ip-10-0-135-159 kubenswrapper[2557]: E0715 19:59:36.813476 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls podName:b78b0486-62b8-4cf0-97d8-fd8682d6604a nodeName:}" failed. No retries permitted until 2026-07-15 19:59:52.813459925 +0000 UTC m=+85.685453355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sfd9" (UID: "b78b0486-62b8-4cf0-97d8-fd8682d6604a") : secret "insights-runtime-extractor-tls" not found Jul 15 19:59:36.813776 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:36.813717 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b78b0486-62b8-4cf0-97d8-fd8682d6604a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:37.116575 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:37.116500 2557 scope.go:117] "RemoveContainer" containerID="c9fe28c4a40961e7006d3c74f3c46563cca36a987317e205eae24089c54041e4" Jul 15 19:59:38.598940 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.598884 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mdxkm" event={"ID":"186e9fb1-120a-4998-8d07-1cf46a96d6eb","Type":"ContainerStarted","Data":"32563bc7758d3f1d8af2c1cd9715f73896dcaaa1d517b256a1d5b85fab1f7fa6"} Jul 15 19:59:38.600771 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.600741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5zsr" event={"ID":"fbcfe517-551b-4abd-9dd0-b1fab5ddb979","Type":"ContainerStarted","Data":"d6fad75a5800523ec432c82c555ad12ba05942a996dbc6eebc6b70f8e4867746"} Jul 15 19:59:38.600916 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.600797 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w5zsr" event={"ID":"fbcfe517-551b-4abd-9dd0-b1fab5ddb979","Type":"ContainerStarted","Data":"0beab470acee4645516833b9daf81184ba38fe5537d695ed0e41f672d4d83d43"} Jul 15 19:59:38.600916 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.600825 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:38.602467 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.602448 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 19:59:38.602601 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.602491 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-b72vx" event={"ID":"7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4","Type":"ContainerStarted","Data":"6cf6be300039280273f2768dcab791f5ecfca46a3ab725c6b42c26b4e42cf211"} Jul 15 19:59:38.602745 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.602724 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:38.620913 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.620873 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mdxkm" podStartSLOduration=65.724786366 podStartE2EDuration="1m10.620859534s" totalImagesPullingTime="4.896073168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:38.618768365 +0000 UTC m=+71.490761797" watchObservedRunningTime="2026-07-15 19:59:38.620859534 +0000 UTC m=+71.492852966" Jul 15 19:59:38.642794 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.642755 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-b72vx" podStartSLOduration=20.23990202 podStartE2EDuration="26.642743628s" totalImagesPullingTime="6.402841608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:38.641174893 +0000 UTC m=+71.513168335" watchObservedRunningTime="2026-07-15 19:59:38.642743628 +0000 UTC m=+71.514737104" Jul 15 19:59:38.666171 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.666133 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w5zsr" podStartSLOduration=33.999731928 podStartE2EDuration="36.666122925s" totalImagesPullingTime="2.666390997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:38.6653068 +0000 UTC m=+71.537300232" watchObservedRunningTime="2026-07-15 19:59:38.666122925 +0000 UTC m=+71.538116355" Jul 15 19:59:38.861176 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:38.861095 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-b72vx" Jul 15 19:59:44.172900 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.172853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:44.175240 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.175210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/099450eb-3cf9-48be-b519-74d00f7f8690-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nlprf\" (UID: \"099450eb-3cf9-48be-b519-74d00f7f8690\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:44.274138 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.274088 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:44.274304 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.274209 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:44.274304 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.274235 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:44.274791 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.274758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7be1a52a-e535-4911-8a11-94a4d0810d99-service-ca-bundle\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:44.276599 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.276577 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5415a430-c08c-4476-b46c-b0a059e27d0b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-v7kjd\" (UID: \"5415a430-c08c-4476-b46c-b0a059e27d0b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:44.276698 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.276607 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be1a52a-e535-4911-8a11-94a4d0810d99-metrics-certs\") pod \"router-default-6d485b46b6-r668j\" (UID: \"7be1a52a-e535-4911-8a11-94a4d0810d99\") " pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:44.415544 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.415515 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nf6mj\"" Jul 15 19:59:44.423157 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.423107 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" Jul 15 19:59:44.464977 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.464946 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wqp6p\"" Jul 15 19:59:44.472722 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.472692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:44.502926 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.502772 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lmcrv\"" Jul 15 19:59:44.510827 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.510804 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" Jul 15 19:59:44.571289 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.571263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf"] Jul 15 19:59:44.649949 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.649930 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d485b46b6-r668j"] Jul 15 19:59:44.651712 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:44.651687 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be1a52a_e535_4911_8a11_94a4d0810d99.slice/crio-2c753a35e631057c3252d18b71250d77a0c3fa6b243dc13dff761b0c80df9de0 WatchSource:0}: Error finding container 2c753a35e631057c3252d18b71250d77a0c3fa6b243dc13dff761b0c80df9de0: Status 404 returned error can't find the container with id 2c753a35e631057c3252d18b71250d77a0c3fa6b243dc13dff761b0c80df9de0 Jul 15 19:59:44.674518 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:44.674441 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd"] Jul 15 19:59:44.675054 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:44.675029 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5415a430_c08c_4476_b46c_b0a059e27d0b.slice/crio-dfb1163ca88a4a10fc54b465e7d7000516be29b546a6f691e369d00d1c6b2f43 WatchSource:0}: Error finding container dfb1163ca88a4a10fc54b465e7d7000516be29b546a6f691e369d00d1c6b2f43: Status 404 returned error can't find the container with id dfb1163ca88a4a10fc54b465e7d7000516be29b546a6f691e369d00d1c6b2f43 Jul 15 19:59:45.625827 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:45.625789 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" event={"ID":"099450eb-3cf9-48be-b519-74d00f7f8690","Type":"ContainerStarted","Data":"f58ac9ec421d169c176723c612acc9497e9310111621fb1eafa4d4fe9aa4de1f"} Jul 15 19:59:45.627134 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:45.627106 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" event={"ID":"5415a430-c08c-4476-b46c-b0a059e27d0b","Type":"ContainerStarted","Data":"dfb1163ca88a4a10fc54b465e7d7000516be29b546a6f691e369d00d1c6b2f43"} Jul 15 19:59:45.628708 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:45.628678 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d485b46b6-r668j" event={"ID":"7be1a52a-e535-4911-8a11-94a4d0810d99","Type":"ContainerStarted","Data":"21d06faab65a493c6239e2ca38a6efe55e0f122258c49ba19eca4d13063c626a"} Jul 15 19:59:45.628793 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:45.628723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d485b46b6-r668j" event={"ID":"7be1a52a-e535-4911-8a11-94a4d0810d99","Type":"ContainerStarted","Data":"2c753a35e631057c3252d18b71250d77a0c3fa6b243dc13dff761b0c80df9de0"} Jul 15 19:59:46.473904 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.473878 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:46.477032 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.477015 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:46.503947 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.503898 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d485b46b6-r668j" podStartSLOduration=34.503879485 podStartE2EDuration="34.503879485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:45.652694025 +0000 UTC m=+78.524687475" watchObservedRunningTime="2026-07-15 19:59:46.503879485 +0000 UTC m=+79.375872912" Jul 15 19:59:46.634819 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.634783 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" event={"ID":"5415a430-c08c-4476-b46c-b0a059e27d0b","Type":"ContainerStarted","Data":"6d71fec75d9dc3067f0aa0d646e0bb4a38ef58d07c7108920350b1e0eacbacfb"} Jul 15 19:59:46.635237 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.634904 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:46.636394 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.636370 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d485b46b6-r668j" Jul 15 19:59:46.657332 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:46.657284 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-v7kjd" podStartSLOduration=32.899221089 podStartE2EDuration="34.65726711s" totalImagesPullingTime="1.758046021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:46.655896164 +0000 UTC m=+79.527889610" watchObservedRunningTime="2026-07-15 19:59:46.65726711 +0000 UTC m=+79.529260543" Jul 15 19:59:47.640683 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:47.640653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" event={"ID":"099450eb-3cf9-48be-b519-74d00f7f8690","Type":"ContainerStarted","Data":"fa59b295974704e4475bfdb2e84d9bf478dae0d11156ebe6e46f1d285c20b99b"} Jul 15 19:59:47.641096 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:47.640711 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" event={"ID":"099450eb-3cf9-48be-b519-74d00f7f8690","Type":"ContainerStarted","Data":"ae36cab0aed58960a45be31bb82edf8bfde0ca1f15b50bc9caaa1d48e5a496dd"} Jul 15 19:59:48.607733 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:48.607705 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w5zsr" Jul 15 19:59:48.650751 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:48.650694 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nlprf" podStartSLOduration=34.199020192 podStartE2EDuration="36.65068027s" totalImagesPullingTime="2.451660078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:47.76782716 +0000 UTC m=+80.639820591" watchObservedRunningTime="2026-07-15 19:59:48.65068027 +0000 UTC m=+81.522673702" Jul 15 19:59:52.842585 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:52.842477 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:52.844710 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:52.844689 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b78b0486-62b8-4cf0-97d8-fd8682d6604a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sfd9\" (UID: \"b78b0486-62b8-4cf0-97d8-fd8682d6604a\") " pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:53.092334 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:53.092303 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v5glv\"" Jul 15 19:59:53.099754 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:53.099702 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7sfd9" Jul 15 19:59:53.266975 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:53.266937 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7sfd9"] Jul 15 19:59:53.269693 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:53.269670 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78b0486_62b8_4cf0_97d8_fd8682d6604a.slice/crio-d2ad9ab446d127d7b56178da0c5add5179158eeee79bd71de288547555bcad85 WatchSource:0}: Error finding container d2ad9ab446d127d7b56178da0c5add5179158eeee79bd71de288547555bcad85: Status 404 returned error can't find the container with id d2ad9ab446d127d7b56178da0c5add5179158eeee79bd71de288547555bcad85 Jul 15 19:59:53.660985 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:53.660955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sfd9" event={"ID":"b78b0486-62b8-4cf0-97d8-fd8682d6604a","Type":"ContainerStarted","Data":"d2ad9ab446d127d7b56178da0c5add5179158eeee79bd71de288547555bcad85"} Jul 15 19:59:55.108390 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.108354 2557 patch_prober.go:28] interesting pod/image-registry-6884f584f6-xx7cb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 19:59:55.108796 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.108406 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 19:59:55.352326 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.352295 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 19:59:55.358715 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.358644 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.361653 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.361630 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 19:59:55.361982 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.361956 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 19:59:55.362142 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.362008 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 19:59:55.362142 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.362086 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 19:59:55.362999 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.362975 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 19:59:55.362999 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.362993 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 19:59:55.363209 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.363064 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kj6kq\"" Jul 15 19:59:55.364391 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.364362 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 19:59:55.368520 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.368501 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 19:59:55.371340 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.371321 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 19:59:55.466776 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.466742 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.466923 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.466796 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.466923 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.466861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.467016 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.466944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.467016 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.466992 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9ks\" (UniqueName: \"kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.467127 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.467038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.467190 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.467174 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568188 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568347 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568198 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568347 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568347 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568347 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568271 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9ks\" (UniqueName: \"kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568347 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568310 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.568610 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.568379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.569151 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.569104 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.569151 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.569124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.569151 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.569133 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.569437 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.569413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.570932 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.570903 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.571344 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.571326 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.578521 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.578480 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9ks\" (UniqueName: \"kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks\") pod \"console-65748db6c5-n2lhx\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:55.670210 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:55.670144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 19:59:56.592145 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:56.592117 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 19:59:58.366916 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:58.366884 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 19:59:58.370238 ip-10-0-135-159 kubenswrapper[2557]: W0715 19:59:58.370214 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cd8529_b06e_4cbf_91d9_69912d245c67.slice/crio-01f3cf770ac3cb0e346790fe70aabf33dfc9165f285925b0377397b33fa5759a WatchSource:0}: Error finding container 01f3cf770ac3cb0e346790fe70aabf33dfc9165f285925b0377397b33fa5759a: Status 404 returned error can't find the container with id 01f3cf770ac3cb0e346790fe70aabf33dfc9165f285925b0377397b33fa5759a Jul 15 19:59:58.682769 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:58.682732 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sfd9" event={"ID":"b78b0486-62b8-4cf0-97d8-fd8682d6604a","Type":"ContainerStarted","Data":"d1c2d0cddce573cae40fe37da06e10be3f2e2402e79e80f0fe6754e8da3c970f"} Jul 15 19:59:58.684262 ip-10-0-135-159 kubenswrapper[2557]: I0715 19:59:58.684228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65748db6c5-n2lhx" event={"ID":"a7cd8529-b06e-4cbf-91d9-69912d245c67","Type":"ContainerStarted","Data":"01f3cf770ac3cb0e346790fe70aabf33dfc9165f285925b0377397b33fa5759a"} Jul 15 20:00:03.714271 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:03.714183 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sfd9" event={"ID":"b78b0486-62b8-4cf0-97d8-fd8682d6604a","Type":"ContainerStarted","Data":"9cbf56ea1ad86a9512d7846868aa6fed5ab014b9a25c170bc69d270dfbb792a8"} Jul 15 20:00:03.716434 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:03.716377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65748db6c5-n2lhx" event={"ID":"a7cd8529-b06e-4cbf-91d9-69912d245c67","Type":"ContainerStarted","Data":"4dd76b543a7a432448b64e938be8305d5afc0b8f0c28b9dde96f8886ce0f6445"} Jul 15 20:00:03.740776 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:03.740725 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65748db6c5-n2lhx" podStartSLOduration=4.294835031 podStartE2EDuration="8.740708422s" totalImagesPullingTime="4.445873391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:03.739308844 +0000 UTC m=+96.611302278" watchObservedRunningTime="2026-07-15 20:00:03.740708422 +0000 UTC m=+96.612701832" Jul 15 20:00:05.585858 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:05.585816 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6q45l" Jul 15 20:00:05.670394 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:05.670362 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:00:05.670567 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:05.670428 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:00:05.676739 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:05.676713 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:00:05.729844 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:05.729816 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:00:06.731720 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:06.731684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sfd9" event={"ID":"b78b0486-62b8-4cf0-97d8-fd8682d6604a","Type":"ContainerStarted","Data":"8f45c96c9ae903bf49683a12cbe659e71be27821609bb903e3291f3d03dbbc7b"} Jul 15 20:00:06.756304 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:06.756260 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7sfd9" podStartSLOduration=33.585066831 podStartE2EDuration="46.756244967s" totalImagesPullingTime="13.171178136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:59:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:06.754235347 +0000 UTC m=+99.626228780" watchObservedRunningTime="2026-07-15 20:00:06.756244967 +0000 UTC m=+99.628238398" Jul 15 20:00:12.458298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:12.458265 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 20:00:17.747829 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.747793 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk"] Jul 15 20:00:17.751505 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.751488 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.755804 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.755785 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 20:00:17.757192 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.757176 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 20:00:17.757274 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.757211 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 20:00:17.757339 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.757294 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dqq48\"" Jul 15 20:00:17.769461 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.769440 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk"] Jul 15 20:00:17.791602 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.791579 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ps4m9"] Jul 15 20:00:17.793793 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.793776 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.798146 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.798129 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f5fs6\"" Jul 15 20:00:17.798238 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.798147 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 20:00:17.798238 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.798129 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 20:00:17.799553 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.799534 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 20:00:17.851725 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851690 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.851828 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a10514e-4a04-4c14-affb-39370ff32e41-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.851828 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-wtmp\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.851898 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.851898 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-textfile\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.851898 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851885 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.851988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851910 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfg64\" (UniqueName: \"kubernetes.io/projected/dc291c77-b425-499b-bb76-b5226720249e-kube-api-access-mfg64\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.851988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-root\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.851988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.851971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-sys\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.852107 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.852010 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-metrics-client-ca\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.852107 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.852042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.852107 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.852064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmlzs\" (UniqueName: \"kubernetes.io/projected/1a10514e-4a04-4c14-affb-39370ff32e41-kube-api-access-nmlzs\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.852201 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.852125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-tls\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.952780 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-textfile\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952827 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfg64\" (UniqueName: \"kubernetes.io/projected/dc291c77-b425-499b-bb76-b5226720249e-kube-api-access-mfg64\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-root\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-sys\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:17.952898 2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 20:00:17.952925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-metrics-client-ca\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952935 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952983 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-root\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.952991 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-sys\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:17.953030 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls podName:1a10514e-4a04-4c14-affb-39370ff32e41 nodeName:}" failed. No retries permitted until 2026-07-15 20:00:18.452942306 +0000 UTC m=+111.324935719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-wldzk" (UID: "1a10514e-4a04-4c14-affb-39370ff32e41") : secret "openshift-state-metrics-tls" not found Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmlzs\" (UniqueName: \"kubernetes.io/projected/1a10514e-4a04-4c14-affb-39370ff32e41-kube-api-access-nmlzs\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-tls\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953294 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a10514e-4a04-4c14-affb-39370ff32e41-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.953653 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953340 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-textfile\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953653 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953344 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-wtmp\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953653 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953455 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-wtmp\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953795 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953701 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-metrics-client-ca\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.953902 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.953879 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.954019 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.954000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a10514e-4a04-4c14-affb-39370ff32e41-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.955509 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.955487 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.955602 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.955489 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:17.955697 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.955675 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc291c77-b425-499b-bb76-b5226720249e-node-exporter-tls\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.980983 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.980961 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfg64\" (UniqueName: \"kubernetes.io/projected/dc291c77-b425-499b-bb76-b5226720249e-kube-api-access-mfg64\") pod \"node-exporter-ps4m9\" (UID: \"dc291c77-b425-499b-bb76-b5226720249e\") " pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:17.981254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:17.981224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmlzs\" (UniqueName: \"kubernetes.io/projected/1a10514e-4a04-4c14-affb-39370ff32e41-kube-api-access-nmlzs\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:18.102562 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.102527 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ps4m9" Jul 15 20:00:18.113436 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:18.113410 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc291c77_b425_499b_bb76_b5226720249e.slice/crio-e0e3fa1af75fbf1a18e905904a587b692dc8daea8cb63954d610bcd32181a298 WatchSource:0}: Error finding container e0e3fa1af75fbf1a18e905904a587b692dc8daea8cb63954d610bcd32181a298: Status 404 returned error can't find the container with id e0e3fa1af75fbf1a18e905904a587b692dc8daea8cb63954d610bcd32181a298 Jul 15 20:00:18.457002 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.456906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:18.459310 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.459285 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a10514e-4a04-4c14-affb-39370ff32e41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wldzk\" (UID: \"1a10514e-4a04-4c14-affb-39370ff32e41\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:18.660446 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.660415 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" Jul 15 20:00:18.774701 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.774655 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ps4m9" event={"ID":"dc291c77-b425-499b-bb76-b5226720249e","Type":"ContainerStarted","Data":"e0e3fa1af75fbf1a18e905904a587b692dc8daea8cb63954d610bcd32181a298"} Jul 15 20:00:18.815261 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.815238 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk"] Jul 15 20:00:18.832602 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.832574 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:00:18.835130 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.835112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.839446 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.839422 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 20:00:18.839544 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.839423 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 20:00:18.839638 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.839623 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fbg6q\"" Jul 15 20:00:18.839743 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.839720 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 20:00:18.840630 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.840612 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 20:00:18.841157 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.841137 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 20:00:18.841277 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.841265 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 20:00:18.845571 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.845552 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 20:00:18.845657 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.845635 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 20:00:18.845717 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.845659 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 20:00:18.860699 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860675 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.860801 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.860801 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.860921 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860865 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.860921 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzhbn\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861023 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860948 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861023 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.860987 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861149 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861025 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861149 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861149 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861118 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861149 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861142 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861331 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861179 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.861331 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.861225 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.875218 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.875199 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:00:18.962543 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962507 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhbn\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.962708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962705 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.962925 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.963357 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:18.963044 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle podName:42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c nodeName:}" failed. No retries permitted until 2026-07-15 20:00:19.463020368 +0000 UTC m=+112.335013788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c") : configmap references non-existent config key: ca-bundle.crt Jul 15 20:00:18.963497 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.963467 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.964097 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.963926 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.966510 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.966436 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.966510 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.966438 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.967171 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.967108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.967742 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.967722 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.968008 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.967978 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.968532 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.968512 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.969342 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.969325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.969434 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.969354 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.969723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.969698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:18.982828 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:18.982802 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzhbn\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:19.019141 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:19.019118 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a10514e_4a04_4c14_affb_39370ff32e41.slice/crio-ae08b19157d8104ec33f9bdf3cba327042c5f7329ba169e04dcaf04f9102e82c WatchSource:0}: Error finding container ae08b19157d8104ec33f9bdf3cba327042c5f7329ba169e04dcaf04f9102e82c: Status 404 returned error can't find the container with id ae08b19157d8104ec33f9bdf3cba327042c5f7329ba169e04dcaf04f9102e82c Jul 15 20:00:19.467521 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.467490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:19.468215 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.468195 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:19.746254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.746178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:00:19.780835 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.780799 2557 generic.go:356] "Generic (PLEG): container finished" podID="dc291c77-b425-499b-bb76-b5226720249e" containerID="ccdd3c063e448e3ffa7e3cd2f0a3b0e37c1c9536316addb8c49eddfd677d07e7" exitCode=0 Jul 15 20:00:19.781235 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.780888 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ps4m9" event={"ID":"dc291c77-b425-499b-bb76-b5226720249e","Type":"ContainerDied","Data":"ccdd3c063e448e3ffa7e3cd2f0a3b0e37c1c9536316addb8c49eddfd677d07e7"} Jul 15 20:00:19.783142 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.783114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" event={"ID":"1a10514e-4a04-4c14-affb-39370ff32e41","Type":"ContainerStarted","Data":"aa3e6b2e575af467e758a5f0c151c8c9d7bff91d76d45f9361926c3806a9fc4c"} Jul 15 20:00:19.783248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.783174 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" event={"ID":"1a10514e-4a04-4c14-affb-39370ff32e41","Type":"ContainerStarted","Data":"75097b1fe262fec81e24e7d61892815f7554fd4da7dbbbcc5c80e1b0cbad38d8"} Jul 15 20:00:19.783248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.783206 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" event={"ID":"1a10514e-4a04-4c14-affb-39370ff32e41","Type":"ContainerStarted","Data":"ae08b19157d8104ec33f9bdf3cba327042c5f7329ba169e04dcaf04f9102e82c"} Jul 15 20:00:19.905926 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:19.905899 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:00:19.909741 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:19.909717 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f4a0a0_fd8b_4f3c_a087_e75d24d2e03c.slice/crio-d5fa7663a5b5f03b1b01736e7d0fcff285ad1fd17a00b67fb773336343728bcd WatchSource:0}: Error finding container d5fa7663a5b5f03b1b01736e7d0fcff285ad1fd17a00b67fb773336343728bcd: Status 404 returned error can't find the container with id d5fa7663a5b5f03b1b01736e7d0fcff285ad1fd17a00b67fb773336343728bcd Jul 15 20:00:20.789732 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.789698 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" event={"ID":"1a10514e-4a04-4c14-affb-39370ff32e41","Type":"ContainerStarted","Data":"fec4506bf9175bbf59dc46c8068b33f75b67ec726a547e7c072dac4723d6dec5"} Jul 15 20:00:20.792125 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.792100 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ps4m9" event={"ID":"dc291c77-b425-499b-bb76-b5226720249e","Type":"ContainerStarted","Data":"bcd78d5c9ef28d658bb116b69faf4c8b8f0ba297db445232af1c5f560fc156bf"} Jul 15 20:00:20.792246 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.792142 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ps4m9" event={"ID":"dc291c77-b425-499b-bb76-b5226720249e","Type":"ContainerStarted","Data":"71ac84b91c8f307c87421bd2e986405ceb4b55f10930acb187ec17c3d80e972f"} Jul 15 20:00:20.793356 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.793334 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"d5fa7663a5b5f03b1b01736e7d0fcff285ad1fd17a00b67fb773336343728bcd"} Jul 15 20:00:20.812806 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.812756 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wldzk" podStartSLOduration=2.5414534460000002 podStartE2EDuration="3.812741415s" totalImagesPullingTime="1.271287969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:20.811257176 +0000 UTC m=+113.683250609" watchObservedRunningTime="2026-07-15 20:00:20.812741415 +0000 UTC m=+113.684734873" Jul 15 20:00:20.835427 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:20.835388 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ps4m9" podStartSLOduration=2.832397205 podStartE2EDuration="3.83537412s" totalImagesPullingTime="928.528125ms" totalInitContainerRuntime="74.44879ms" isStatefulPod=true podCreationTimestamp="2026-07-15 20:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:20.834493106 +0000 UTC m=+113.706486564" watchObservedRunningTime="2026-07-15 20:00:20.83537412 +0000 UTC m=+113.707367543" Jul 15 20:00:21.798133 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:21.798096 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e" exitCode=0 Jul 15 20:00:21.798505 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:21.798165 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e"} Jul 15 20:00:22.456606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.456570 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d"] Jul 15 20:00:22.458599 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.458583 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.461809 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:22.461780 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"monitoring-plugin-5f68df7f58-s4j9d\" is forbidden: User \"system:node:ip-10-0-135-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-159.ec2.internal' and this object" podUID="64042cf5-0a53-4c15-8c85-e9522bbc4f25" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.493315 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.493276 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9852m\" (UniqueName: \"kubernetes.io/projected/64042cf5-0a53-4c15-8c85-e9522bbc4f25-kube-api-access-9852m\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.493453 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:22.493330 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"monitoring-plugin-dockercfg-m2qdf\" is forbidden: User \"system:node:ip-10-0-135-159.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-159.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m2qdf\"" type="*v1.Secret" Jul 15 20:00:22.493453 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.493385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64042cf5-0a53-4c15-8c85-e9522bbc4f25-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.494853 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:22.494822 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"monitoring-plugin-cert\" is forbidden: User \"system:node:ip-10-0-135-159.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-159.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" type="*v1.Secret" Jul 15 20:00:22.497499 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.497479 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d"] Jul 15 20:00:22.594640 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.594612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9852m\" (UniqueName: \"kubernetes.io/projected/64042cf5-0a53-4c15-8c85-e9522bbc4f25-kube-api-access-9852m\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.594753 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.594677 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64042cf5-0a53-4c15-8c85-e9522bbc4f25-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:22.603176 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:22.603154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9852m\" (UniqueName: \"kubernetes.io/projected/64042cf5-0a53-4c15-8c85-e9522bbc4f25-kube-api-access-9852m\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:23.018502 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.018467 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-597ffb665-qkwl6"] Jul 15 20:00:23.021373 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.021348 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.025327 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025255 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 20:00:23.025327 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025285 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 20:00:23.025516 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025290 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 20:00:23.025516 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025349 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 20:00:23.025516 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025364 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 20:00:23.025882 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.025862 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gn6bn\"" Jul 15 20:00:23.030670 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.030648 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 20:00:23.038280 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.038258 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597ffb665-qkwl6"] Jul 15 20:00:23.099815 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.099782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q47w\" (UniqueName: \"kubernetes.io/projected/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-kube-api-access-6q47w\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.099849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.099879 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-serving-certs-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.099936 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-metrics-client-ca\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.099979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.100039 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100290 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.100066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.100290 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.100159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-federate-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201342 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201305 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q47w\" (UniqueName: \"kubernetes.io/projected/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-kube-api-access-6q47w\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-serving-certs-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201441 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-metrics-client-ca\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201720 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201720 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.201720 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.201597 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-federate-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.202797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.202721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-serving-certs-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.202797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.202725 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-metrics-client-ca\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.203227 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.203178 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.204643 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.204615 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.204813 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.204656 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-secret-telemeter-client\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.204813 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.204625 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-federate-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.204955 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.204934 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-telemeter-client-tls\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.211662 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.211628 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q47w\" (UniqueName: \"kubernetes.io/projected/b6c72266-38cd-4dc7-9ec8-dc28672aef2b-kube-api-access-6q47w\") pod \"telemeter-client-597ffb665-qkwl6\" (UID: \"b6c72266-38cd-4dc7-9ec8-dc28672aef2b\") " pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.333482 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.333324 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" Jul 15 20:00:23.500861 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.497855 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597ffb665-qkwl6"] Jul 15 20:00:23.504115 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:23.504059 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c72266_38cd_4dc7_9ec8_dc28672aef2b.slice/crio-4da5598da23d2afc9e7d2b69b755f19c5638f7d6ce96f4a29e528f739113e2ca WatchSource:0}: Error finding container 4da5598da23d2afc9e7d2b69b755f19c5638f7d6ce96f4a29e528f739113e2ca: Status 404 returned error can't find the container with id 4da5598da23d2afc9e7d2b69b755f19c5638f7d6ce96f4a29e528f739113e2ca Jul 15 20:00:23.547629 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.547604 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 20:00:23.557424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.557403 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64042cf5-0a53-4c15-8c85-e9522bbc4f25-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s4j9d\" (UID: \"64042cf5-0a53-4c15-8c85-e9522bbc4f25\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:23.807132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.807093 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" event={"ID":"b6c72266-38cd-4dc7-9ec8-dc28672aef2b","Type":"ContainerStarted","Data":"4da5598da23d2afc9e7d2b69b755f19c5638f7d6ce96f4a29e528f739113e2ca"} Jul 15 20:00:23.809728 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.809704 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618"} Jul 15 20:00:23.809858 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.809771 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0"} Jul 15 20:00:23.809858 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.809812 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9"} Jul 15 20:00:23.809858 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.809848 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257"} Jul 15 20:00:23.809988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:23.809886 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5"} Jul 15 20:00:24.087124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.087023 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m2qdf\"" Jul 15 20:00:24.088032 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.088013 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:24.254958 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.254931 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d"] Jul 15 20:00:24.350855 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:24.350784 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64042cf5_0a53_4c15_8c85_e9522bbc4f25.slice/crio-87a8deea55cef8ccc3d58e8de7866081b32904762112904c27c18d32ae7746c3 WatchSource:0}: Error finding container 87a8deea55cef8ccc3d58e8de7866081b32904762112904c27c18d32ae7746c3: Status 404 returned error can't find the container with id 87a8deea55cef8ccc3d58e8de7866081b32904762112904c27c18d32ae7746c3 Jul 15 20:00:24.815520 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.815461 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" event={"ID":"64042cf5-0a53-4c15-8c85-e9522bbc4f25","Type":"ContainerStarted","Data":"87a8deea55cef8ccc3d58e8de7866081b32904762112904c27c18d32ae7746c3"} Jul 15 20:00:24.819292 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.819260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerStarted","Data":"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c"} Jul 15 20:00:24.859780 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:24.859730 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.389198382 podStartE2EDuration="6.859715891s" totalImagesPullingTime="3.450312851s" totalInitContainerRuntime="20.204658ms" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:24.858657561 +0000 UTC m=+117.730650992" watchObservedRunningTime="2026-07-15 20:00:24.859715891 +0000 UTC m=+117.731709323" Jul 15 20:00:26.832798 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.832757 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" event={"ID":"b6c72266-38cd-4dc7-9ec8-dc28672aef2b","Type":"ContainerStarted","Data":"29d03b06acbcb035ebd43d573ffac7db01f6e5615c3db24ddbd71fc7d87b199f"} Jul 15 20:00:26.833248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.832841 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" event={"ID":"b6c72266-38cd-4dc7-9ec8-dc28672aef2b","Type":"ContainerStarted","Data":"ea863c889a595d49b346edd01912068788629244b6f665dc303558c95c7abc00"} Jul 15 20:00:26.833248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.832868 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" event={"ID":"b6c72266-38cd-4dc7-9ec8-dc28672aef2b","Type":"ContainerStarted","Data":"0fed4f7c0f4f87e432b0536ae073b9b39dd40fee6079d4498ba8fb6173b46210"} Jul 15 20:00:26.834145 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.834123 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" event={"ID":"64042cf5-0a53-4c15-8c85-e9522bbc4f25","Type":"ContainerStarted","Data":"d67757ec6f55e23e511001f4782c5fc025ec306452841078e0f25d44fda7db8b"} Jul 15 20:00:26.834330 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.834315 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:26.843852 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.843832 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" Jul 15 20:00:26.865298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.865250 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-597ffb665-qkwl6" podStartSLOduration=2.464861347 podStartE2EDuration="4.865237464s" totalImagesPullingTime="2.400376117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:26.863990506 +0000 UTC m=+119.735983930" watchObservedRunningTime="2026-07-15 20:00:26.865237464 +0000 UTC m=+119.737230896" Jul 15 20:00:26.884994 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:26.884958 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s4j9d" podStartSLOduration=3.332180534 podStartE2EDuration="4.884951144s" totalImagesPullingTime="1.55277061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:26.882392922 +0000 UTC m=+119.754386356" watchObservedRunningTime="2026-07-15 20:00:26.884951144 +0000 UTC m=+119.756944637" Jul 15 20:00:30.851307 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.851275 2557 generic.go:356] "Generic (PLEG): container finished" podID="e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8" containerID="dbd672ee9edcd9a79249247306b4252a91f53ae1a3edf74cefa8e62f27b932c7" exitCode=0 Jul 15 20:00:30.851719 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.851350 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" event={"ID":"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8","Type":"ContainerDied","Data":"dbd672ee9edcd9a79249247306b4252a91f53ae1a3edf74cefa8e62f27b932c7"} Jul 15 20:00:30.851719 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.851676 2557 scope.go:117] "RemoveContainer" containerID="dbd672ee9edcd9a79249247306b4252a91f53ae1a3edf74cefa8e62f27b932c7" Jul 15 20:00:30.852921 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.852899 2557 generic.go:356] "Generic (PLEG): container finished" podID="046bd0a4-42a9-4c26-a779-dd08c9aa8d4b" containerID="21e04a1b98325866c785a3f22451bcec7fa4b1fc865e736c4e5542c05e05aca4" exitCode=0 Jul 15 20:00:30.853013 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.852979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" event={"ID":"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b","Type":"ContainerDied","Data":"21e04a1b98325866c785a3f22451bcec7fa4b1fc865e736c4e5542c05e05aca4"} Jul 15 20:00:30.853313 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.853294 2557 scope.go:117] "RemoveContainer" containerID="21e04a1b98325866c785a3f22451bcec7fa4b1fc865e736c4e5542c05e05aca4" Jul 15 20:00:30.854436 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.854415 2557 generic.go:356] "Generic (PLEG): container finished" podID="bacfee8f-3f1b-4686-915e-8971f0d072c5" containerID="e05972985328b99967b552083d28c3a57d6e07c38bc91f52b0e77a819a6d8a25" exitCode=0 Jul 15 20:00:30.854500 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.854458 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" event={"ID":"bacfee8f-3f1b-4686-915e-8971f0d072c5","Type":"ContainerDied","Data":"e05972985328b99967b552083d28c3a57d6e07c38bc91f52b0e77a819a6d8a25"} Jul 15 20:00:30.854688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:30.854677 2557 scope.go:117] "RemoveContainer" containerID="e05972985328b99967b552083d28c3a57d6e07c38bc91f52b0e77a819a6d8a25" Jul 15 20:00:31.860595 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:31.860559 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pg599" event={"ID":"bacfee8f-3f1b-4686-915e-8971f0d072c5","Type":"ContainerStarted","Data":"7947e3acb809509c5840b3eae3603ceb47ae118cfa5a60334f6b359c0d4feef8"} Jul 15 20:00:31.862464 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:31.862439 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-hpnzf" event={"ID":"e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8","Type":"ContainerStarted","Data":"32527c9b09af93bd70ee073d8d1532503a1d3cdda996c7683d7ac213ad1bb028"} Jul 15 20:00:31.864174 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:31.864151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rhkwb" event={"ID":"046bd0a4-42a9-4c26-a779-dd08c9aa8d4b","Type":"ContainerStarted","Data":"714c005c9d32f58b9ee6f3e21d7141b5f0531ba1603eceff751be40df5ddf935"} Jul 15 20:00:36.819817 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:36.819781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 20:00:36.822087 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:36.822047 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1840d153-4cbf-447c-992f-2c01012eacbe-metrics-certs\") pod \"network-metrics-daemon-dfwrk\" (UID: \"1840d153-4cbf-447c-992f-2c01012eacbe\") " pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 20:00:36.842787 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:36.842762 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kl6c8\"" Jul 15 20:00:36.850274 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:36.850260 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfwrk" Jul 15 20:00:36.991100 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:36.991061 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dfwrk"] Jul 15 20:00:36.992863 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:00:36.992834 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1840d153_4cbf_447c_992f_2c01012eacbe.slice/crio-3329f9c0bbc89b55f877a4e6dd3753151798a201e53e7fd3b78d871cc77e3eeb WatchSource:0}: Error finding container 3329f9c0bbc89b55f877a4e6dd3753151798a201e53e7fd3b78d871cc77e3eeb: Status 404 returned error can't find the container with id 3329f9c0bbc89b55f877a4e6dd3753151798a201e53e7fd3b78d871cc77e3eeb Jul 15 20:00:37.478065 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.478002 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerName="registry" containerID="cri-o://01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77" gracePeriod=30 Jul 15 20:00:37.708341 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.708312 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 20:00:37.827766 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.827746 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828101 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.827779 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828101 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.827819 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828101 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.827928 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ntjf\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828101 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.827990 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828101 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828028 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828366 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828105 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828366 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828149 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration\") pod \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\" (UID: \"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5\") " Jul 15 20:00:37.828366 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828213 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:00:37.828524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828400 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-certificates\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.828638 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.828609 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:00:37.830513 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.830483 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf" (OuterVolumeSpecName: "kube-api-access-9ntjf") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "kube-api-access-9ntjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:00:37.830765 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.830720 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:00:37.830848 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.830749 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:00:37.830848 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.830829 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:00:37.831162 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.831135 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:00:37.836875 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.836851 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" (UID: "5c41ff42-68be-46ed-9ad5-685fe9e7e8b5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:00:37.888247 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.888223 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfwrk" event={"ID":"1840d153-4cbf-447c-992f-2c01012eacbe","Type":"ContainerStarted","Data":"3329f9c0bbc89b55f877a4e6dd3753151798a201e53e7fd3b78d871cc77e3eeb"} Jul 15 20:00:37.889423 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.889402 2557 generic.go:356] "Generic (PLEG): container finished" podID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerID="01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77" exitCode=0 Jul 15 20:00:37.889487 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.889450 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" Jul 15 20:00:37.889521 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.889486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" event={"ID":"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5","Type":"ContainerDied","Data":"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77"} Jul 15 20:00:37.889553 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.889531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884f584f6-xx7cb" event={"ID":"5c41ff42-68be-46ed-9ad5-685fe9e7e8b5","Type":"ContainerDied","Data":"d686af85598b41c722c5c5001bd6feee23894a4bbb16b9d76f0e19e2f3933bbb"} Jul 15 20:00:37.889585 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.889552 2557 scope.go:117] "RemoveContainer" containerID="01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77" Jul 15 20:00:37.898623 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.898607 2557 scope.go:117] "RemoveContainer" containerID="01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77" Jul 15 20:00:37.898871 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:00:37.898853 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77\": container with ID starting with 01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77 not found: ID does not exist" containerID="01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77" Jul 15 20:00:37.898926 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.898878 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77"} err="failed to get container status \"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77\": rpc error: code = NotFound desc = could not find container \"01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77\": container with ID starting with 01be8088bbaaf418e873e38a5f6f3f5662b243342db0cdbe3ee1f33cebd4cb77 not found: ID does not exist" Jul 15 20:00:37.929606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929585 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-installation-pull-secrets\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929604 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-registry-tls\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929614 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ntjf\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-kube-api-access-9ntjf\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929623 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-ca-trust-extracted\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929631 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-bound-sa-token\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929640 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-trusted-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.929723 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.929648 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5-image-registry-private-configuration\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:00:37.932702 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.932681 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 20:00:37.946995 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:37.946976 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6884f584f6-xx7cb"] Jul 15 20:00:38.121251 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:38.121199 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" path="/var/lib/kubelet/pods/5c41ff42-68be-46ed-9ad5-685fe9e7e8b5/volumes" Jul 15 20:00:38.774836 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:38.774799 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 20:00:39.901447 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:39.901412 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfwrk" event={"ID":"1840d153-4cbf-447c-992f-2c01012eacbe","Type":"ContainerStarted","Data":"4ce4ce28e47562921dc57d858ee962a1cacc1253274e5490a8adef164d6748fd"} Jul 15 20:00:39.901818 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:00:39.901464 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfwrk" event={"ID":"1840d153-4cbf-447c-992f-2c01012eacbe","Type":"ContainerStarted","Data":"41307236ad6f4f75f5c54ead0fe72b4b6476e4b3dcb4bedf2263b94d9ef25b97"} Jul 15 20:01:03.797033 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:03.796961 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65748db6c5-n2lhx" podUID="a7cd8529-b06e-4cbf-91d9-69912d245c67" containerName="console" containerID="cri-o://4dd76b543a7a432448b64e938be8305d5afc0b8f0c28b9dde96f8886ce0f6445" gracePeriod=15 Jul 15 20:01:04.013398 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.013367 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cd8529-b06e-4cbf-91d9-69912d245c67" containerID="4dd76b543a7a432448b64e938be8305d5afc0b8f0c28b9dde96f8886ce0f6445" exitCode=0 Jul 15 20:01:04.013540 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.013447 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65748db6c5-n2lhx" event={"ID":"a7cd8529-b06e-4cbf-91d9-69912d245c67","Type":"ContainerDied","Data":"4dd76b543a7a432448b64e938be8305d5afc0b8f0c28b9dde96f8886ce0f6445"} Jul 15 20:01:04.045264 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.045241 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:01:04.075707 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.075608 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dfwrk" podStartSLOduration=154.119550538 podStartE2EDuration="2m36.075591946s" totalImagesPullingTime="1.956041408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:39.920593519 +0000 UTC m=+132.792586952" watchObservedRunningTime="2026-07-15 20:01:04.075591946 +0000 UTC m=+156.947585377" Jul 15 20:01:04.233227 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233190 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233255 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233278 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233311 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt9ks\" (UniqueName: \"kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233326 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233371 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233625 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233424 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config\") pod \"a7cd8529-b06e-4cbf-91d9-69912d245c67\" (UID: \"a7cd8529-b06e-4cbf-91d9-69912d245c67\") " Jul 15 20:01:04.233753 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233623 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:04.233821 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233723 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config" (OuterVolumeSpecName: "console-config") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:04.233883 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233814 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca" (OuterVolumeSpecName: "service-ca") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:04.233883 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.233832 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:04.235770 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.235743 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:04.235770 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.235762 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:04.235889 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.235807 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks" (OuterVolumeSpecName: "kube-api-access-nt9ks") pod "a7cd8529-b06e-4cbf-91d9-69912d245c67" (UID: "a7cd8529-b06e-4cbf-91d9-69912d245c67"). InnerVolumeSpecName "kube-api-access-nt9ks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334352 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-oauth-serving-cert\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334377 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334386 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334395 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt9ks\" (UniqueName: \"kubernetes.io/projected/a7cd8529-b06e-4cbf-91d9-69912d245c67-kube-api-access-nt9ks\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334403 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cd8529-b06e-4cbf-91d9-69912d245c67-trusted-ca-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334411 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-serving-cert\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:04.334415 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:04.334420 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7cd8529-b06e-4cbf-91d9-69912d245c67-console-oauth-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:05.019729 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:05.019698 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65748db6c5-n2lhx" Jul 15 20:01:05.020132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:05.019708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65748db6c5-n2lhx" event={"ID":"a7cd8529-b06e-4cbf-91d9-69912d245c67","Type":"ContainerDied","Data":"01f3cf770ac3cb0e346790fe70aabf33dfc9165f285925b0377397b33fa5759a"} Jul 15 20:01:05.020132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:05.019830 2557 scope.go:117] "RemoveContainer" containerID="4dd76b543a7a432448b64e938be8305d5afc0b8f0c28b9dde96f8886ce0f6445" Jul 15 20:01:05.046012 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:05.045982 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 20:01:05.052249 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:05.052224 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65748db6c5-n2lhx"] Jul 15 20:01:06.120992 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:06.120960 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7cd8529-b06e-4cbf-91d9-69912d245c67" path="/var/lib/kubelet/pods/a7cd8529-b06e-4cbf-91d9-69912d245c67/volumes" Jul 15 20:01:38.310298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310265 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310538 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="alertmanager" containerID="cri-o://ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" gracePeriod=120 Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310627 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy" containerID="cri-o://0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" gracePeriod=120 Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310617 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-web" containerID="cri-o://fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" gracePeriod=120 Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310653 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="config-reloader" containerID="cri-o://a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" gracePeriod=120 Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310612 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-metric" containerID="cri-o://e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" gracePeriod=120 Jul 15 20:01:38.310775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:38.310711 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="prom-label-proxy" containerID="cri-o://d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" gracePeriod=120 Jul 15 20:01:39.094190 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094162 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:01:39.094504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094487 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7cd8529-b06e-4cbf-91d9-69912d245c67" containerName="console" Jul 15 20:01:39.094504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094502 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7cd8529-b06e-4cbf-91d9-69912d245c67" containerName="console" Jul 15 20:01:39.094642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094526 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerName="registry" Jul 15 20:01:39.094642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094535 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerName="registry" Jul 15 20:01:39.094642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094609 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7cd8529-b06e-4cbf-91d9-69912d245c67" containerName="console" Jul 15 20:01:39.094642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.094618 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c41ff42-68be-46ed-9ad5-685fe9e7e8b5" containerName="registry" Jul 15 20:01:39.096797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.096780 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.104637 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.104603 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 20:01:39.105874 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.105856 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kj6kq\"" Jul 15 20:01:39.105874 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.105869 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 20:01:39.106001 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.105885 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 20:01:39.106001 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.105868 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 20:01:39.106181 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.106161 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 20:01:39.106225 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.106187 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 20:01:39.106225 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.106220 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 20:01:39.114834 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.114812 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:01:39.120331 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.120306 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 20:01:39.152631 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152604 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" exitCode=0 Jul 15 20:01:39.152631 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152632 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" exitCode=0 Jul 15 20:01:39.152767 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152643 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" exitCode=0 Jul 15 20:01:39.152767 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152652 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" exitCode=0 Jul 15 20:01:39.152767 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152683 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c"} Jul 15 20:01:39.152861 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152767 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0"} Jul 15 20:01:39.152861 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152797 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257"} Jul 15 20:01:39.152861 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.152828 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5"} Jul 15 20:01:39.202123 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbg9l\" (UniqueName: \"kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202219 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202219 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202219 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202208 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202325 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202325 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202264 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.202325 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.202308 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303545 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303579 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbg9l\" (UniqueName: \"kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.303919 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.303670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.304291 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.304264 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.304461 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.304433 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.304461 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.304450 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.304598 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.304490 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.306151 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.306131 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.306341 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.306322 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.316318 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.316299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbg9l\" (UniqueName: \"kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l\") pod \"console-6bf95f7756-7xdf4\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.405393 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.405367 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:39.577765 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.577740 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:39.596056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.596026 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:01:39.599243 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:01:39.599220 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac74618_fe58_430c_9d1e_5ffefee69bde.slice/crio-63236ab7080bf84a1e4babae539327e8fb2d9bc8bff33cfffcb366ea830ec1c7 WatchSource:0}: Error finding container 63236ab7080bf84a1e4babae539327e8fb2d9bc8bff33cfffcb366ea830ec1c7: Status 404 returned error can't find the container with id 63236ab7080bf84a1e4babae539327e8fb2d9bc8bff33cfffcb366ea830ec1c7 Jul 15 20:01:39.717191 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717116 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717191 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717152 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717191 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717186 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717210 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717228 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717249 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzhbn\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717312 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717343 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717377 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717403 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717432 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717458 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:01:39.717894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717478 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config\") pod \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\" (UID: \"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c\") " Jul 15 20:01:39.717894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717518 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:39.717894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717726 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-main-db\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.717894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.717746 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.719116 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.719087 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:01:39.720027 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.719999 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume" (OuterVolumeSpecName: "config-volume") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.720424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.720399 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out" (OuterVolumeSpecName: "config-out") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:01:39.720512 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.720416 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:01:39.720512 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.720431 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.720690 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.720665 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.720805 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.720781 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.721590 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.721546 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.721846 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.721789 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn" (OuterVolumeSpecName: "kube-api-access-tzhbn") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "kube-api-access-tzhbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:01:39.726255 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.726228 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.732293 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.732273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config" (OuterVolumeSpecName: "web-config") pod "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" (UID: "42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:01:39.818197 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818171 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818197 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818193 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-main-tls\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818203 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-tls-assets\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818211 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzhbn\" (UniqueName: \"kubernetes.io/projected/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-kube-api-access-tzhbn\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818221 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-out\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818229 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818237 2557 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-config-volume\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818246 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-metrics-client-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818253 2557 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-cluster-tls-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818262 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:39.818323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:39.818271 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c-web-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:01:40.158647 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.158610 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf95f7756-7xdf4" event={"ID":"1ac74618-fe58-430c-9d1e-5ffefee69bde","Type":"ContainerStarted","Data":"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7"} Jul 15 20:01:40.158789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.158661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf95f7756-7xdf4" event={"ID":"1ac74618-fe58-430c-9d1e-5ffefee69bde","Type":"ContainerStarted","Data":"63236ab7080bf84a1e4babae539327e8fb2d9bc8bff33cfffcb366ea830ec1c7"} Jul 15 20:01:40.161543 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161519 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" exitCode=0 Jul 15 20:01:40.161543 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161541 2557 generic.go:356] "Generic (PLEG): container finished" podID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerID="fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" exitCode=0 Jul 15 20:01:40.161688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618"} Jul 15 20:01:40.161688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161643 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9"} Jul 15 20:01:40.161688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161589 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.161688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161680 2557 scope.go:117] "RemoveContainer" containerID="d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" Jul 15 20:01:40.161842 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.161670 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c","Type":"ContainerDied","Data":"d5fa7663a5b5f03b1b01736e7d0fcff285ad1fd17a00b67fb773336343728bcd"} Jul 15 20:01:40.169391 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.169373 2557 scope.go:117] "RemoveContainer" containerID="e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" Jul 15 20:01:40.176467 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.176450 2557 scope.go:117] "RemoveContainer" containerID="0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" Jul 15 20:01:40.184840 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.184809 2557 scope.go:117] "RemoveContainer" containerID="fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" Jul 15 20:01:40.191550 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.191511 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bf95f7756-7xdf4" podStartSLOduration=1.191497885 podStartE2EDuration="1.191497885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:40.190398756 +0000 UTC m=+193.062392189" watchObservedRunningTime="2026-07-15 20:01:40.191497885 +0000 UTC m=+193.063491317" Jul 15 20:01:40.192020 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.192004 2557 scope.go:117] "RemoveContainer" containerID="a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" Jul 15 20:01:40.198798 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.198782 2557 scope.go:117] "RemoveContainer" containerID="ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" Jul 15 20:01:40.205706 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.205689 2557 scope.go:117] "RemoveContainer" containerID="dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e" Jul 15 20:01:40.212619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.212597 2557 scope.go:117] "RemoveContainer" containerID="d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" Jul 15 20:01:40.212880 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.212857 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c\": container with ID starting with d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c not found: ID does not exist" containerID="d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" Jul 15 20:01:40.212943 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.212891 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c"} err="failed to get container status \"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c\": rpc error: code = NotFound desc = could not find container \"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c\": container with ID starting with d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c not found: ID does not exist" Jul 15 20:01:40.212943 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.212912 2557 scope.go:117] "RemoveContainer" containerID="e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" Jul 15 20:01:40.213221 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.213199 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618\": container with ID starting with e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618 not found: ID does not exist" containerID="e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" Jul 15 20:01:40.213282 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213226 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618"} err="failed to get container status \"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618\": rpc error: code = NotFound desc = could not find container \"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618\": container with ID starting with e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618 not found: ID does not exist" Jul 15 20:01:40.213282 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213242 2557 scope.go:117] "RemoveContainer" containerID="0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" Jul 15 20:01:40.213510 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.213493 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0\": container with ID starting with 0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0 not found: ID does not exist" containerID="0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" Jul 15 20:01:40.213628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213519 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0"} err="failed to get container status \"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0\": rpc error: code = NotFound desc = could not find container \"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0\": container with ID starting with 0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0 not found: ID does not exist" Jul 15 20:01:40.213628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213543 2557 scope.go:117] "RemoveContainer" containerID="fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" Jul 15 20:01:40.213789 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.213773 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9\": container with ID starting with fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9 not found: ID does not exist" containerID="fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" Jul 15 20:01:40.213831 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213795 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9"} err="failed to get container status \"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9\": rpc error: code = NotFound desc = could not find container \"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9\": container with ID starting with fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9 not found: ID does not exist" Jul 15 20:01:40.213831 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.213811 2557 scope.go:117] "RemoveContainer" containerID="a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" Jul 15 20:01:40.213998 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.213982 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257\": container with ID starting with a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257 not found: ID does not exist" containerID="a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" Jul 15 20:01:40.214034 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214000 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257"} err="failed to get container status \"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257\": rpc error: code = NotFound desc = could not find container \"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257\": container with ID starting with a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257 not found: ID does not exist" Jul 15 20:01:40.214034 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214013 2557 scope.go:117] "RemoveContainer" containerID="ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" Jul 15 20:01:40.214237 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.214218 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5\": container with ID starting with ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5 not found: ID does not exist" containerID="ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" Jul 15 20:01:40.214298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214246 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5"} err="failed to get container status \"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5\": rpc error: code = NotFound desc = could not find container \"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5\": container with ID starting with ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5 not found: ID does not exist" Jul 15 20:01:40.214298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214267 2557 scope.go:117] "RemoveContainer" containerID="dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e" Jul 15 20:01:40.214484 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:01:40.214467 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e\": container with ID starting with dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e not found: ID does not exist" containerID="dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e" Jul 15 20:01:40.214524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214490 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e"} err="failed to get container status \"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e\": rpc error: code = NotFound desc = could not find container \"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e\": container with ID starting with dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e not found: ID does not exist" Jul 15 20:01:40.214524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214505 2557 scope.go:117] "RemoveContainer" containerID="d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c" Jul 15 20:01:40.214690 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214668 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c"} err="failed to get container status \"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c\": rpc error: code = NotFound desc = could not find container \"d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c\": container with ID starting with d7b0762dbe91979b81471342448f48bd3c046611d575e438e03a7b7b390cd91c not found: ID does not exist" Jul 15 20:01:40.214734 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214692 2557 scope.go:117] "RemoveContainer" containerID="e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618" Jul 15 20:01:40.214881 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214866 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618"} err="failed to get container status \"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618\": rpc error: code = NotFound desc = could not find container \"e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618\": container with ID starting with e1bea72769881dbb9dd36f706b9af951e9c267f24e1658eb16590f97606e3618 not found: ID does not exist" Jul 15 20:01:40.214918 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.214881 2557 scope.go:117] "RemoveContainer" containerID="0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0" Jul 15 20:01:40.215131 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215104 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0"} err="failed to get container status \"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0\": rpc error: code = NotFound desc = could not find container \"0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0\": container with ID starting with 0fff564743b2d28fe8f1a6a82b4d4152e2367ff5989d00e9604c4daafa5067f0 not found: ID does not exist" Jul 15 20:01:40.215131 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215132 2557 scope.go:117] "RemoveContainer" containerID="fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9" Jul 15 20:01:40.215362 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215344 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9"} err="failed to get container status \"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9\": rpc error: code = NotFound desc = could not find container \"fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9\": container with ID starting with fcbd7c4ea04df5fa301f02a0828d9feb21d5e28ff914195e89288d59a6a9cea9 not found: ID does not exist" Jul 15 20:01:40.215362 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215361 2557 scope.go:117] "RemoveContainer" containerID="a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257" Jul 15 20:01:40.215574 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215557 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257"} err="failed to get container status \"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257\": rpc error: code = NotFound desc = could not find container \"a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257\": container with ID starting with a62aa9a7422d84d7dd57f7b996bf1a9ec51e449e21973cd4ed552cba57670257 not found: ID does not exist" Jul 15 20:01:40.215617 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215575 2557 scope.go:117] "RemoveContainer" containerID="ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5" Jul 15 20:01:40.215765 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215750 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5"} err="failed to get container status \"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5\": rpc error: code = NotFound desc = could not find container \"ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5\": container with ID starting with ab7b8be8ae0d76e7915b7feca24ee97d9fbf5b3a214a64fd8580feeea95ac0b5 not found: ID does not exist" Jul 15 20:01:40.215805 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215764 2557 scope.go:117] "RemoveContainer" containerID="dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e" Jul 15 20:01:40.215946 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.215931 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e"} err="failed to get container status \"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e\": rpc error: code = NotFound desc = could not find container \"dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e\": container with ID starting with dccabb0d0b00561ebb790f962a35f7f89195d8084a51cc8df61e6c2b722fd79e not found: ID does not exist" Jul 15 20:01:40.223008 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.222988 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:40.231246 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.231224 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:40.293132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293102 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:40.293448 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293434 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-web" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293453 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-web" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293466 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="alertmanager" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293473 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="alertmanager" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293484 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="prom-label-proxy" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293490 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="prom-label-proxy" Jul 15 20:01:40.293496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293497 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="init-config-reloader" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293503 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="init-config-reloader" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293509 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-metric" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293516 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-metric" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293528 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293536 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293547 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="config-reloader" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293552 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="config-reloader" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293613 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="prom-label-proxy" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293626 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293632 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="config-reloader" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293638 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="alertmanager" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293643 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-web" Jul 15 20:01:40.293666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.293650 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" containerName="kube-rbac-proxy-metric" Jul 15 20:01:40.297310 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.297295 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.304460 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.304437 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 20:01:40.304824 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.304804 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 20:01:40.305118 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305097 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 20:01:40.305118 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305114 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 20:01:40.305291 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305205 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 20:01:40.305386 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305371 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fbg6q\"" Jul 15 20:01:40.305437 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305420 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 20:01:40.305499 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.305436 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 20:01:40.307442 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.307423 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 20:01:40.314015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.313998 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 20:01:40.326815 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.326795 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:40.427568 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427536 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427573 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-web-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427635 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427679 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-out\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427729 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.427866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.428057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427867 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52bbd\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-kube-api-access-52bbd\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.428057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.428057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.427902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529301 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529264 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529360 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52bbd\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-kube-api-access-52bbd\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529392 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529449 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-web-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529520 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-out\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.529785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.529580 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.530227 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.530202 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.530346 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.530239 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.530419 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.530382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.532729 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.532686 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533311 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533311 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533111 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533311 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533311 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533292 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533509 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533656 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533641 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.533691 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-web-config\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.534002 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.533986 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-config-out\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.540827 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.540809 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52bbd\" (UniqueName: \"kubernetes.io/projected/5e26c5aa-58e2-4298-8f4b-42bb118d2dd7-kube-api-access-52bbd\") pod \"alertmanager-main-0\" (UID: \"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.606351 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.606325 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 20:01:40.749715 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:40.749573 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 20:01:40.752118 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:01:40.752092 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e26c5aa_58e2_4298_8f4b_42bb118d2dd7.slice/crio-5bfe83119537778c52b756d381c7c9b5b035c44fbe9464dcc34b3717ac86517a WatchSource:0}: Error finding container 5bfe83119537778c52b756d381c7c9b5b035c44fbe9464dcc34b3717ac86517a: Status 404 returned error can't find the container with id 5bfe83119537778c52b756d381c7c9b5b035c44fbe9464dcc34b3717ac86517a Jul 15 20:01:41.167744 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:41.167706 2557 generic.go:356] "Generic (PLEG): container finished" podID="5e26c5aa-58e2-4298-8f4b-42bb118d2dd7" containerID="c43c61f1278b055239a5e05d282a941fa50389751c12a81001241294886480a1" exitCode=0 Jul 15 20:01:41.167904 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:41.167805 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerDied","Data":"c43c61f1278b055239a5e05d282a941fa50389751c12a81001241294886480a1"} Jul 15 20:01:41.167904 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:41.167887 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"5bfe83119537778c52b756d381c7c9b5b035c44fbe9464dcc34b3717ac86517a"} Jul 15 20:01:42.122463 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.122428 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c" path="/var/lib/kubelet/pods/42f4a0a0-fd8b-4f3c-a087-e75d24d2e03c/volumes" Jul 15 20:01:42.175266 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175232 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"bd317392418b22d30ac02405c00039d115568c1f9c858411a24275f59718dccc"} Jul 15 20:01:42.175416 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175296 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"ae2347e20f5170ce7a12db70adc985eaa1affd28ca81af25484e1c54f7b9147e"} Jul 15 20:01:42.175416 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175322 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"5bb13b47ab75a76a08c7486126e21cc12dc54ff0b421f7fe6443e7c52c184023"} Jul 15 20:01:42.175416 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175345 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"d288383f92775ecef0aa6c66d53b5b4fea62cbc0375a1449a6402f696431519d"} Jul 15 20:01:42.175416 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175369 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"a198ada531098108794d9689fc9c4615f993cfb25c1fc935b2b9aaf1bd57dfa0"} Jul 15 20:01:42.175416 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.175393 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e26c5aa-58e2-4298-8f4b-42bb118d2dd7","Type":"ContainerStarted","Data":"8b8762d5a1f99397b01e928cbb83522a992dfaf134875cb8fb98d5cfbf904f6b"} Jul 15 20:01:42.210085 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:42.210022 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.191305352 podStartE2EDuration="2.210006651s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.701299ms" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:42.207316597 +0000 UTC m=+195.079310042" watchObservedRunningTime="2026-07-15 20:01:42.210006651 +0000 UTC m=+195.082000083" Jul 15 20:01:49.405703 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:49.405658 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:49.406289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:49.405740 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:49.410453 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:49.410431 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:01:50.211683 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:01:50.211650 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:03:27.957273 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:03:27.957240 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:03:27.960955 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:03:27.960925 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:03:27.963942 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:03:27.963920 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:03:27.967737 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:03:27.967720 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:03:27.967955 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:03:27.967940 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 20:04:06.582158 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.582124 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2"] Jul 15 20:04:06.585588 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.585561 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.592830 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.592785 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 20:04:06.592830 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.592804 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 20:04:06.593002 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.592890 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmwsx\"" Jul 15 20:04:06.613490 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.613459 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2"] Jul 15 20:04:06.688220 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.688178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.688220 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.688229 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.688448 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.688291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ngc8\" (UniqueName: \"kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.788898 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.788856 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.789131 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.788908 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.789131 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.788932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ngc8\" (UniqueName: \"kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.789335 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.789310 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.789409 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.789358 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.800011 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.799985 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ngc8\" (UniqueName: \"kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:06.894986 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:06.894897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:07.031484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:07.031458 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2"] Jul 15 20:04:07.033925 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:04:07.033898 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e500e57_e05e_4e49_aef8_f7a5263eeb8f.slice/crio-c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7 WatchSource:0}: Error finding container c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7: Status 404 returned error can't find the container with id c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7 Jul 15 20:04:07.035936 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:07.035918 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:04:07.748611 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:07.748575 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" event={"ID":"0e500e57-e05e-4e49-aef8-f7a5263eeb8f","Type":"ContainerStarted","Data":"c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7"} Jul 15 20:04:13.786215 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:13.786181 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerID="98f347d0bbd88b4b59edf3bf213d4520cce7a42728ac40274f20300e97a5148a" exitCode=0 Jul 15 20:04:13.786713 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:13.786275 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" event={"ID":"0e500e57-e05e-4e49-aef8-f7a5263eeb8f","Type":"ContainerDied","Data":"98f347d0bbd88b4b59edf3bf213d4520cce7a42728ac40274f20300e97a5148a"} Jul 15 20:04:16.800991 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:16.800956 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerID="bbbbb120314cdb4463029daf242711904928cd633307eaf60fa074d27283729e" exitCode=0 Jul 15 20:04:16.801398 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:16.801027 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" event={"ID":"0e500e57-e05e-4e49-aef8-f7a5263eeb8f","Type":"ContainerDied","Data":"bbbbb120314cdb4463029daf242711904928cd633307eaf60fa074d27283729e"} Jul 15 20:04:23.834784 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:23.834750 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerID="af9e791d49a86de7a65a818b985ea2bb4a3d40484a6cb513f6da429783e6a41b" exitCode=0 Jul 15 20:04:23.835167 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:23.834839 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" event={"ID":"0e500e57-e05e-4e49-aef8-f7a5263eeb8f","Type":"ContainerDied","Data":"af9e791d49a86de7a65a818b985ea2bb4a3d40484a6cb513f6da429783e6a41b"} Jul 15 20:04:24.968663 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:24.968638 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:25.050373 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.050330 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util\") pod \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " Jul 15 20:04:25.050567 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.050438 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle\") pod \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " Jul 15 20:04:25.050567 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.050460 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ngc8\" (UniqueName: \"kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8\") pod \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\" (UID: \"0e500e57-e05e-4e49-aef8-f7a5263eeb8f\") " Jul 15 20:04:25.051147 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.051120 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle" (OuterVolumeSpecName: "bundle") pod "0e500e57-e05e-4e49-aef8-f7a5263eeb8f" (UID: "0e500e57-e05e-4e49-aef8-f7a5263eeb8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:25.052624 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.052602 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8" (OuterVolumeSpecName: "kube-api-access-7ngc8") pod "0e500e57-e05e-4e49-aef8-f7a5263eeb8f" (UID: "0e500e57-e05e-4e49-aef8-f7a5263eeb8f"). InnerVolumeSpecName "kube-api-access-7ngc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:04:25.054727 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.054702 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util" (OuterVolumeSpecName: "util") pod "0e500e57-e05e-4e49-aef8-f7a5263eeb8f" (UID: "0e500e57-e05e-4e49-aef8-f7a5263eeb8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:25.152032 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.151932 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:25.152032 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.151970 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ngc8\" (UniqueName: \"kubernetes.io/projected/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-kube-api-access-7ngc8\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:25.152032 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.151981 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e500e57-e05e-4e49-aef8-f7a5263eeb8f-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:25.844805 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.844766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" event={"ID":"0e500e57-e05e-4e49-aef8-f7a5263eeb8f","Type":"ContainerDied","Data":"c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7"} Jul 15 20:04:25.844976 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.844824 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0d3b3307f050a29fb56b8bb444991e482b27b444b5af76217cc144c015fe1b7" Jul 15 20:04:25.844976 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:25.844789 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754044bc2" Jul 15 20:04:30.890183 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890146 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6"] Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890474 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="pull" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890484 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="pull" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890492 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="extract" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890498 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="extract" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890507 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="util" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890513 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="util" Jul 15 20:04:30.890619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.890573 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e500e57-e05e-4e49-aef8-f7a5263eeb8f" containerName="extract" Jul 15 20:04:30.922039 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.922005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:30.926527 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.926502 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-m7slm\"" Jul 15 20:04:30.927387 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.927365 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 20:04:30.927520 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.927406 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:04:30.928700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:30.928674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6"] Jul 15 20:04:31.001311 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.001273 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5rc\" (UniqueName: \"kubernetes.io/projected/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-kube-api-access-xn5rc\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.001484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.001355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.102262 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.102223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5rc\" (UniqueName: \"kubernetes.io/projected/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-kube-api-access-xn5rc\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.102455 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.102320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.102703 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.102681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.111481 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.111456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5rc\" (UniqueName: \"kubernetes.io/projected/5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66-kube-api-access-xn5rc\") pod \"cert-manager-operator-controller-manager-55c444bf86-85sv6\" (UID: \"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.232688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.232588 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" Jul 15 20:04:31.412864 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.412769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6"] Jul 15 20:04:31.417061 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:04:31.417033 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3a6cb8_c0d6_4de9_81af_10b7bd94ff66.slice/crio-00cd8465567c74e5da942a5b5a6f72a4377172da9571ee1ff35fdb476a17e869 WatchSource:0}: Error finding container 00cd8465567c74e5da942a5b5a6f72a4377172da9571ee1ff35fdb476a17e869: Status 404 returned error can't find the container with id 00cd8465567c74e5da942a5b5a6f72a4377172da9571ee1ff35fdb476a17e869 Jul 15 20:04:31.871559 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:31.871524 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" event={"ID":"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66","Type":"ContainerStarted","Data":"00cd8465567c74e5da942a5b5a6f72a4377172da9571ee1ff35fdb476a17e869"} Jul 15 20:04:33.882910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:33.882858 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" event={"ID":"5a3a6cb8-c0d6-4de9-81af-10b7bd94ff66","Type":"ContainerStarted","Data":"6402c368b84fded4755098b8da4d0f14cf3fa941b98fd4e0cfdae87e21a94637"} Jul 15 20:04:33.921133 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:33.921059 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85sv6" podStartSLOduration=1.596950547 podStartE2EDuration="3.921034352s" totalImagesPullingTime="2.324083805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:33.91796516 +0000 UTC m=+366.789958593" watchObservedRunningTime="2026-07-15 20:04:33.921034352 +0000 UTC m=+366.793027784" Jul 15 20:04:34.788626 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.788594 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x"] Jul 15 20:04:34.791059 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.791034 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.793217 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.793188 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x"] Jul 15 20:04:34.795219 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.795198 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 20:04:34.795342 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.795222 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 20:04:34.796254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.796234 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmwsx\"" Jul 15 20:04:34.835862 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.835818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.836025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.835875 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.836025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.836014 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdhll\" (UniqueName: \"kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.936665 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.936625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.937048 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.936681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.937048 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.936796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdhll\" (UniqueName: \"kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.937170 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.937141 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.937206 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.937153 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:34.976519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:34.976479 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdhll\" (UniqueName: \"kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:35.101249 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:35.101165 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:35.256525 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:35.256480 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x"] Jul 15 20:04:35.270029 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:04:35.269999 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc1a68c_a7ce_443a_95e8_19a1d6a1353c.slice/crio-cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4 WatchSource:0}: Error finding container cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4: Status 404 returned error can't find the container with id cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4 Jul 15 20:04:35.892811 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:35.892772 2557 generic.go:356] "Generic (PLEG): container finished" podID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerID="dc27cfb190c1883f52885331208a5054af7100f60e48783428abb2df9359240d" exitCode=0 Jul 15 20:04:35.893018 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:35.892827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" event={"ID":"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c","Type":"ContainerDied","Data":"dc27cfb190c1883f52885331208a5054af7100f60e48783428abb2df9359240d"} Jul 15 20:04:35.893018 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:35.892878 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" event={"ID":"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c","Type":"ContainerStarted","Data":"cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4"} Jul 15 20:04:38.390966 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.390928 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs"] Jul 15 20:04:38.393031 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.393012 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.395865 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.395839 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 20:04:38.395979 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.395870 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 20:04:38.397324 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.397305 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-d67ms\"" Jul 15 20:04:38.406688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.406660 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs"] Jul 15 20:04:38.467923 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.467892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr8ws\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-kube-api-access-xr8ws\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.468097 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.467949 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.568913 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.568878 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr8ws\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-kube-api-access-xr8ws\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.569145 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.568947 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.578395 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.578365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.578579 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.578532 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr8ws\" (UniqueName: \"kubernetes.io/projected/70e652c5-a264-4b5a-a4a5-60603284877d-kube-api-access-xr8ws\") pod \"cert-manager-cainjector-559cdd8c87-jlhjs\" (UID: \"70e652c5-a264-4b5a-a4a5-60603284877d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.774534 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.774501 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" Jul 15 20:04:38.908440 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.908405 2557 generic.go:356] "Generic (PLEG): container finished" podID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerID="f711ccfb7c617b99dd94491847022aaad96668c450ebc85a53cdd2fb0517ec00" exitCode=0 Jul 15 20:04:38.908637 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.908486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" event={"ID":"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c","Type":"ContainerDied","Data":"f711ccfb7c617b99dd94491847022aaad96668c450ebc85a53cdd2fb0517ec00"} Jul 15 20:04:38.911535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:38.911459 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs"] Jul 15 20:04:38.913869 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:04:38.913847 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70e652c5_a264_4b5a_a4a5_60603284877d.slice/crio-10c04dcd5f813d2812085035ffb4a5c74da2b73dc75a42f972b8fa185e13528c WatchSource:0}: Error finding container 10c04dcd5f813d2812085035ffb4a5c74da2b73dc75a42f972b8fa185e13528c: Status 404 returned error can't find the container with id 10c04dcd5f813d2812085035ffb4a5c74da2b73dc75a42f972b8fa185e13528c Jul 15 20:04:39.915176 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:39.915140 2557 generic.go:356] "Generic (PLEG): container finished" podID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerID="1a1fcabc6244e7ed7931546c4d89929ca8e857826e89649cf2db56946ef6e825" exitCode=0 Jul 15 20:04:39.915620 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:39.915187 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" event={"ID":"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c","Type":"ContainerDied","Data":"1a1fcabc6244e7ed7931546c4d89929ca8e857826e89649cf2db56946ef6e825"} Jul 15 20:04:39.916359 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:39.916337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" event={"ID":"70e652c5-a264-4b5a-a4a5-60603284877d","Type":"ContainerStarted","Data":"10c04dcd5f813d2812085035ffb4a5c74da2b73dc75a42f972b8fa185e13528c"} Jul 15 20:04:41.047115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.047084 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:41.094324 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.094287 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle\") pod \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " Jul 15 20:04:41.094525 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.094373 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdhll\" (UniqueName: \"kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll\") pod \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " Jul 15 20:04:41.094525 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.094416 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util\") pod \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\" (UID: \"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c\") " Jul 15 20:04:41.094790 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.094762 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle" (OuterVolumeSpecName: "bundle") pod "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" (UID: "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:41.096493 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.096463 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll" (OuterVolumeSpecName: "kube-api-access-vdhll") pod "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" (UID: "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c"). InnerVolumeSpecName "kube-api-access-vdhll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:04:41.101850 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.101824 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util" (OuterVolumeSpecName: "util") pod "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" (UID: "3cc1a68c-a7ce-443a-95e8-19a1d6a1353c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:41.195015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.194940 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:41.195015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.194972 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdhll\" (UniqueName: \"kubernetes.io/projected/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-kube-api-access-vdhll\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:41.195015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.194982 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc1a68c-a7ce-443a-95e8-19a1d6a1353c-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:41.928001 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.927962 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" event={"ID":"3cc1a68c-a7ce-443a-95e8-19a1d6a1353c","Type":"ContainerDied","Data":"cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4"} Jul 15 20:04:41.928187 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.928021 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd14856964024900736374d0e30f341d6f2ad9ef31fb71d89b1b9dce19cc66d4" Jul 15 20:04:41.928187 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:41.928041 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx958x" Jul 15 20:04:43.939319 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:43.939283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" event={"ID":"70e652c5-a264-4b5a-a4a5-60603284877d","Type":"ContainerStarted","Data":"47f97248b78b4a65fc120c5408f5a643d5e11f53ba604ea84ea49822aa744b11"} Jul 15 20:04:43.961711 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:43.961651 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jlhjs" podStartSLOduration=1.797170182 podStartE2EDuration="5.961634541s" totalImagesPullingTime="4.164464359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:43.9604766 +0000 UTC m=+376.832470032" watchObservedRunningTime="2026-07-15 20:04:43.961634541 +0000 UTC m=+376.833627972" Jul 15 20:04:45.345135 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345099 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n2k5n"] Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345406 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="pull" Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345416 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="pull" Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345438 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="extract" Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345443 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="extract" Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345460 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="util" Jul 15 20:04:45.345504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345466 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="util" Jul 15 20:04:45.345692 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.345514 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc1a68c-a7ce-443a-95e8-19a1d6a1353c" containerName="extract" Jul 15 20:04:45.347317 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.347299 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.351113 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.351088 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8t4zk\"" Jul 15 20:04:45.369057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.369026 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n2k5n"] Jul 15 20:04:45.434244 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.434213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qkcl\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-kube-api-access-9qkcl\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.434412 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.434294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-bound-sa-token\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.535372 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.535334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-bound-sa-token\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.535570 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.535498 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qkcl\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-kube-api-access-9qkcl\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.547457 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.547429 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qkcl\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-kube-api-access-9qkcl\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.547867 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.547849 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/508fcda5-d266-4cc6-aee9-b3577f640951-bound-sa-token\") pod \"cert-manager-55b4d954d5-n2k5n\" (UID: \"508fcda5-d266-4cc6-aee9-b3577f640951\") " pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.656524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.656427 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n2k5n" Jul 15 20:04:45.787997 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.787966 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n2k5n"] Jul 15 20:04:45.950065 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.949964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n2k5n" event={"ID":"508fcda5-d266-4cc6-aee9-b3577f640951","Type":"ContainerStarted","Data":"f07e8d4d398f64a29315a19ee12a1073964be0e19bcd445f924abdeb3a2540e0"} Jul 15 20:04:45.950065 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.950013 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n2k5n" event={"ID":"508fcda5-d266-4cc6-aee9-b3577f640951","Type":"ContainerStarted","Data":"8030347b91542fc9ee140f8747888c8c7b36e30a43ec4f778fec21aa8d8ff3ed"} Jul 15 20:04:45.980420 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:45.980359 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n2k5n" podStartSLOduration=0.980341037 podStartE2EDuration="980.341037ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:45.977186741 +0000 UTC m=+378.849180173" watchObservedRunningTime="2026-07-15 20:04:45.980341037 +0000 UTC m=+378.852334468" Jul 15 20:04:53.223148 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.223094 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5"] Jul 15 20:04:53.225852 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.225835 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.229225 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.229200 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmwsx\"" Jul 15 20:04:53.229332 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.229262 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 20:04:53.230493 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.230475 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 20:04:53.239192 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.239170 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5"] Jul 15 20:04:53.301934 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.301895 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.302132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.301943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn2pf\" (UniqueName: \"kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.302132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.301972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.402753 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.402716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.402856 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.402768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn2pf\" (UniqueName: \"kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.402856 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.402813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.403257 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.403233 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.403301 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.403242 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.412796 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.412772 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn2pf\" (UniqueName: \"kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.535260 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.535230 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:53.676924 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.676895 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5"] Jul 15 20:04:53.679520 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:04:53.679490 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47840a4_368c_454e_ab18_f5b568906fd2.slice/crio-01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192 WatchSource:0}: Error finding container 01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192: Status 404 returned error can't find the container with id 01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192 Jul 15 20:04:53.986544 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.986459 2557 generic.go:356] "Generic (PLEG): container finished" podID="d47840a4-368c-454e-ab18-f5b568906fd2" containerID="57e92c69384041654f287c974bfa136db06cf209eba163a628e8be553fd42372" exitCode=0 Jul 15 20:04:53.986695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.986537 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" event={"ID":"d47840a4-368c-454e-ab18-f5b568906fd2","Type":"ContainerDied","Data":"57e92c69384041654f287c974bfa136db06cf209eba163a628e8be553fd42372"} Jul 15 20:04:53.986695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:53.986589 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" event={"ID":"d47840a4-368c-454e-ab18-f5b568906fd2","Type":"ContainerStarted","Data":"01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192"} Jul 15 20:04:54.993691 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:54.993605 2557 generic.go:356] "Generic (PLEG): container finished" podID="d47840a4-368c-454e-ab18-f5b568906fd2" containerID="c2d4096c01e70b0a754f541cf06c1ac3982276815951a59fdfe7424656fda8b0" exitCode=0 Jul 15 20:04:54.994018 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:54.993681 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" event={"ID":"d47840a4-368c-454e-ab18-f5b568906fd2","Type":"ContainerDied","Data":"c2d4096c01e70b0a754f541cf06c1ac3982276815951a59fdfe7424656fda8b0"} Jul 15 20:04:55.999764 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:55.999723 2557 generic.go:356] "Generic (PLEG): container finished" podID="d47840a4-368c-454e-ab18-f5b568906fd2" containerID="a17d33397ff831a2adc01b62b345c44825d0ea3c9d48947d4eb2cf94c0b3380c" exitCode=0 Jul 15 20:04:56.000162 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:55.999810 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" event={"ID":"d47840a4-368c-454e-ab18-f5b568906fd2","Type":"ContainerDied","Data":"a17d33397ff831a2adc01b62b345c44825d0ea3c9d48947d4eb2cf94c0b3380c"} Jul 15 20:04:57.136218 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.136193 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:57.234731 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.234694 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn2pf\" (UniqueName: \"kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf\") pod \"d47840a4-368c-454e-ab18-f5b568906fd2\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " Jul 15 20:04:57.234911 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.234786 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle\") pod \"d47840a4-368c-454e-ab18-f5b568906fd2\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " Jul 15 20:04:57.234911 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.234817 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util\") pod \"d47840a4-368c-454e-ab18-f5b568906fd2\" (UID: \"d47840a4-368c-454e-ab18-f5b568906fd2\") " Jul 15 20:04:57.235594 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.235565 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle" (OuterVolumeSpecName: "bundle") pod "d47840a4-368c-454e-ab18-f5b568906fd2" (UID: "d47840a4-368c-454e-ab18-f5b568906fd2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:57.236979 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.236958 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf" (OuterVolumeSpecName: "kube-api-access-nn2pf") pod "d47840a4-368c-454e-ab18-f5b568906fd2" (UID: "d47840a4-368c-454e-ab18-f5b568906fd2"). InnerVolumeSpecName "kube-api-access-nn2pf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:04:57.240413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.240387 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util" (OuterVolumeSpecName: "util") pod "d47840a4-368c-454e-ab18-f5b568906fd2" (UID: "d47840a4-368c-454e-ab18-f5b568906fd2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:04:57.336267 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.336230 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:57.336267 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.336261 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d47840a4-368c-454e-ab18-f5b568906fd2-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:57.336464 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:57.336276 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn2pf\" (UniqueName: \"kubernetes.io/projected/d47840a4-368c-454e-ab18-f5b568906fd2-kube-api-access-nn2pf\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:04:58.016274 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:58.016235 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" Jul 15 20:04:58.016274 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:58.016253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkjcw5" event={"ID":"d47840a4-368c-454e-ab18-f5b568906fd2","Type":"ContainerDied","Data":"01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192"} Jul 15 20:04:58.016489 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:04:58.016332 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01d599ded9d7efe4b335b9c80a174a40ac373aff51ae88a9cd9176254ec26192" Jul 15 20:05:07.376836 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.376801 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z"] Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377143 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="util" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377155 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="util" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377166 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="extract" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377171 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="extract" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377189 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="pull" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377196 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="pull" Jul 15 20:05:07.377242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.377245 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47840a4-368c-454e-ab18-f5b568906fd2" containerName="extract" Jul 15 20:05:07.379735 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.379716 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.388892 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.388871 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 20:05:07.389139 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.389122 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b8zwh\"" Jul 15 20:05:07.389210 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.389186 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 20:05:07.389300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.389280 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 20:05:07.389359 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.389317 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 20:05:07.433662 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.433622 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z"] Jul 15 20:05:07.532300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.532255 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vpp\" (UniqueName: \"kubernetes.io/projected/f57b7da7-fc22-43fb-b405-59083a751d46-kube-api-access-72vpp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.532300 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.532304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.532515 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.532403 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.532515 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.532477 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.532515 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.532501 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633400 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633400 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633392 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633613 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633613 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vpp\" (UniqueName: \"kubernetes.io/projected/f57b7da7-fc22-43fb-b405-59083a751d46-kube-api-access-72vpp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633613 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633491 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.633914 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.633891 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.635782 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.635758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57b7da7-fc22-43fb-b405-59083a751d46-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.635998 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.635980 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.636039 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.635989 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57b7da7-fc22-43fb-b405-59083a751d46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.655474 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.655442 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vpp\" (UniqueName: \"kubernetes.io/projected/f57b7da7-fc22-43fb-b405-59083a751d46-kube-api-access-72vpp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-tbq8z\" (UID: \"f57b7da7-fc22-43fb-b405-59083a751d46\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.689331 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.689289 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:07.883561 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:07.883478 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z"] Jul 15 20:05:07.891274 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:05:07.891243 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57b7da7_fc22_43fb_b405_59083a751d46.slice/crio-62a6e664355548d98ebda7742718acfc09743e7eb0c068f8ec30f784cdc6b629 WatchSource:0}: Error finding container 62a6e664355548d98ebda7742718acfc09743e7eb0c068f8ec30f784cdc6b629: Status 404 returned error can't find the container with id 62a6e664355548d98ebda7742718acfc09743e7eb0c068f8ec30f784cdc6b629 Jul 15 20:05:08.062017 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:08.061975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" event={"ID":"f57b7da7-fc22-43fb-b405-59083a751d46","Type":"ContainerStarted","Data":"62a6e664355548d98ebda7742718acfc09743e7eb0c068f8ec30f784cdc6b629"} Jul 15 20:05:12.085570 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:12.085526 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" event={"ID":"f57b7da7-fc22-43fb-b405-59083a751d46","Type":"ContainerStarted","Data":"813a4819547f93ee5abe72f37a792c1d926ac13966ea07e97f1b0eea409a5171"} Jul 15 20:05:13.092518 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:13.092479 2557 generic.go:356] "Generic (PLEG): container finished" podID="f57b7da7-fc22-43fb-b405-59083a751d46" containerID="813a4819547f93ee5abe72f37a792c1d926ac13966ea07e97f1b0eea409a5171" exitCode=0 Jul 15 20:05:13.093011 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:13.092561 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" event={"ID":"f57b7da7-fc22-43fb-b405-59083a751d46","Type":"ContainerDied","Data":"813a4819547f93ee5abe72f37a792c1d926ac13966ea07e97f1b0eea409a5171"} Jul 15 20:05:14.099657 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:14.099624 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" event={"ID":"f57b7da7-fc22-43fb-b405-59083a751d46","Type":"ContainerStarted","Data":"7a319ccc1b874a26767c09b9ffb6eb6b2b055ebf59c595579a13f29c2f928cf3"} Jul 15 20:05:14.100035 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:14.099719 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:14.132689 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:14.132643 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" podStartSLOduration=2.544886278 podStartE2EDuration="7.132627727s" totalImagesPullingTime="3.577884845s" totalInitContainerRuntime="1.009856604s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:14.129729532 +0000 UTC m=+407.001722964" watchObservedRunningTime="2026-07-15 20:05:14.132627727 +0000 UTC m=+407.004621159" Jul 15 20:05:25.106272 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.106239 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-tbq8z" Jul 15 20:05:25.954542 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.954512 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-snvtg"] Jul 15 20:05:25.956912 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.956893 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:25.961286 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2hxmd\"" Jul 15 20:05:25.961286 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961281 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 20:05:25.961479 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961304 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:05:25.961479 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 20:05:25.961479 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961423 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 20:05:25.961617 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.961604 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 20:05:25.968569 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:25.968547 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-snvtg"] Jul 15 20:05:26.000645 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.000615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-metrics-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.000785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.000654 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjpxt\" (UniqueName: \"kubernetes.io/projected/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-kube-api-access-sjpxt\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.000785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.000677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-manager-config\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.000785 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.000734 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.101405 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.101375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-metrics-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.101572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.101413 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjpxt\" (UniqueName: \"kubernetes.io/projected/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-kube-api-access-sjpxt\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.101572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.101436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-manager-config\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.101572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.101459 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.102134 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.102103 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-manager-config\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.103829 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.103806 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.103917 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.103881 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-metrics-cert\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.112533 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.112507 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjpxt\" (UniqueName: \"kubernetes.io/projected/4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d-kube-api-access-sjpxt\") pod \"lws-controller-manager-67d9958876-snvtg\" (UID: \"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.266725 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.266629 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:26.408325 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:26.408303 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-snvtg"] Jul 15 20:05:26.416114 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:05:26.411441 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e75d5ff_0fba_4ef5_99d8_55bfe5299c9d.slice/crio-ee846e81237d67508f025d5d7645ec1d01320b746f9d364548fd0a2d5aeca454 WatchSource:0}: Error finding container ee846e81237d67508f025d5d7645ec1d01320b746f9d364548fd0a2d5aeca454: Status 404 returned error can't find the container with id ee846e81237d67508f025d5d7645ec1d01320b746f9d364548fd0a2d5aeca454 Jul 15 20:05:27.160005 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:27.159968 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" event={"ID":"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d","Type":"ContainerStarted","Data":"ee846e81237d67508f025d5d7645ec1d01320b746f9d364548fd0a2d5aeca454"} Jul 15 20:05:28.166143 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:28.166108 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" event={"ID":"4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d","Type":"ContainerStarted","Data":"9538382686edca3d8e8f08f06da9fa43c4182229ebd00492afde8b49bfa069f4"} Jul 15 20:05:28.166573 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:28.166206 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:28.215713 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:28.215661 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" podStartSLOduration=1.558804825 podStartE2EDuration="3.215645093s" totalImagesPullingTime="1.656840268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:28.214862232 +0000 UTC m=+421.086855661" watchObservedRunningTime="2026-07-15 20:05:28.215645093 +0000 UTC m=+421.087638524" Jul 15 20:05:38.042733 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.042697 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d8d6f94db-fts7f"] Jul 15 20:05:38.044995 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.044967 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105097 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105046 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8d6f94db-fts7f"] Jul 15 20:05:38.105269 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105103 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-trusted-ca-bundle\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105269 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-console-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105269 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-oauth-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105385 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-service-ca\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105385 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105323 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105385 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfsg8\" (UniqueName: \"kubernetes.io/projected/d95f1711-64d1-42ec-9795-e479c7e97902-kube-api-access-bfsg8\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.105477 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.105395 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-oauth-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.205927 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.205891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-console-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.205943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-oauth-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.205968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-service-ca\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.205991 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206008 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfsg8\" (UniqueName: \"kubernetes.io/projected/d95f1711-64d1-42ec-9795-e479c7e97902-kube-api-access-bfsg8\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206039 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-oauth-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206093 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-trusted-ca-bundle\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206710 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-console-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206768 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-service-ca\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.206841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.206782 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-oauth-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.207033 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.207013 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95f1711-64d1-42ec-9795-e479c7e97902-trusted-ca-bundle\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.208605 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.208588 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-oauth-config\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.208697 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.208678 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d95f1711-64d1-42ec-9795-e479c7e97902-console-serving-cert\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.241214 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.241184 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfsg8\" (UniqueName: \"kubernetes.io/projected/d95f1711-64d1-42ec-9795-e479c7e97902-kube-api-access-bfsg8\") pod \"console-7d8d6f94db-fts7f\" (UID: \"d95f1711-64d1-42ec-9795-e479c7e97902\") " pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.354321 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.354219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:38.370475 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.370449 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g"] Jul 15 20:05:38.373482 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.373459 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.376374 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.376352 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 20:05:38.376494 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.376438 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hqn7q\"" Jul 15 20:05:38.376573 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.376557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 20:05:38.376732 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.376709 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 20:05:38.377168 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.377151 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 20:05:38.392243 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.392212 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g"] Jul 15 20:05:38.508056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.508018 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8d6f94db-fts7f"] Jul 15 20:05:38.510253 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:05:38.510214 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95f1711_64d1_42ec_9795_e479c7e97902.slice/crio-140602f3f0a52a351c2263229b1c784f77f2d146b40b6697782a7450beed3068 WatchSource:0}: Error finding container 140602f3f0a52a351c2263229b1c784f77f2d146b40b6697782a7450beed3068: Status 404 returned error can't find the container with id 140602f3f0a52a351c2263229b1c784f77f2d146b40b6697782a7450beed3068 Jul 15 20:05:38.510367 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510251 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510367 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510292 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a8457d-85b9-4ac8-948b-916e807a8fd2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510367 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510325 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510367 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510531 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510384 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fcch\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-kube-api-access-6fcch\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510531 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510416 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.510531 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.510489 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611392 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a8457d-85b9-4ac8-948b-916e807a8fd2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611450 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611484 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fcch\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-kube-api-access-6fcch\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.611789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.611549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.612269 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.612212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.613824 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.613788 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.614250 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.614227 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a8457d-85b9-4ac8-948b-916e807a8fd2-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.614584 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.614291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.614584 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.614383 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.626695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.626665 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.626961 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.626938 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fcch\" (UniqueName: \"kubernetes.io/projected/d9a8457d-85b9-4ac8-948b-916e807a8fd2-kube-api-access-6fcch\") pod \"istiod-openshift-gateway-94bb8fbfd-vs94g\" (UID: \"d9a8457d-85b9-4ac8-948b-916e807a8fd2\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.688858 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.688814 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:38.939318 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:38.939220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g"] Jul 15 20:05:38.942334 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:05:38.942299 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a8457d_85b9_4ac8_948b_916e807a8fd2.slice/crio-24ba0560d5a9b021729d47f68e56188ad4c67e930859b52c1915922490f845e3 WatchSource:0}: Error finding container 24ba0560d5a9b021729d47f68e56188ad4c67e930859b52c1915922490f845e3: Status 404 returned error can't find the container with id 24ba0560d5a9b021729d47f68e56188ad4c67e930859b52c1915922490f845e3 Jul 15 20:05:39.172846 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:39.172813 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67d9958876-snvtg" Jul 15 20:05:39.220846 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:39.220725 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" event={"ID":"d9a8457d-85b9-4ac8-948b-916e807a8fd2","Type":"ContainerStarted","Data":"24ba0560d5a9b021729d47f68e56188ad4c67e930859b52c1915922490f845e3"} Jul 15 20:05:39.225167 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:39.225133 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8d6f94db-fts7f" event={"ID":"d95f1711-64d1-42ec-9795-e479c7e97902","Type":"ContainerStarted","Data":"355ab54c92697c647fdf6a43050300db4333fd64c79af9be6dca1576b2c83e29"} Jul 15 20:05:39.225324 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:39.225302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8d6f94db-fts7f" event={"ID":"d95f1711-64d1-42ec-9795-e479c7e97902","Type":"ContainerStarted","Data":"140602f3f0a52a351c2263229b1c784f77f2d146b40b6697782a7450beed3068"} Jul 15 20:05:39.281680 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:39.281632 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d8d6f94db-fts7f" podStartSLOduration=2.28161504 podStartE2EDuration="2.28161504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:39.279560399 +0000 UTC m=+432.151553832" watchObservedRunningTime="2026-07-15 20:05:39.28161504 +0000 UTC m=+432.153608471" Jul 15 20:05:41.171183 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:41.171146 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 20:05:41.171441 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:41.171217 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 20:05:41.237671 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:41.237640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" event={"ID":"d9a8457d-85b9-4ac8-948b-916e807a8fd2","Type":"ContainerStarted","Data":"344b4b953f8eb8a4cf32c503843fb433385b4cb171e9b7d97174b2560597c7d9"} Jul 15 20:05:41.287476 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:41.287421 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" podStartSLOduration=1.060856801 podStartE2EDuration="3.287402424s" totalImagesPullingTime="2.226545623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:41.279266232 +0000 UTC m=+434.151259663" watchObservedRunningTime="2026-07-15 20:05:41.287402424 +0000 UTC m=+434.159395855" Jul 15 20:05:42.242753 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:42.242716 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:42.244675 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:42.244648 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vs94g" Jul 15 20:05:48.355186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:48.355141 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:48.355772 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:48.355213 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:48.360782 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:48.360755 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:49.282309 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:49.282274 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d8d6f94db-fts7f" Jul 15 20:05:49.411516 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:05:49.411479 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:06:14.431917 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.431847 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bf95f7756-7xdf4" podUID="1ac74618-fe58-430c-9d1e-5ffefee69bde" containerName="console" containerID="cri-o://750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7" gracePeriod=15 Jul 15 20:06:14.681233 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.681199 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:06:14.721025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.720940 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.720994 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721287 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721025 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbg9l\" (UniqueName: \"kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721287 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721190 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721287 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721275 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721443 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721342 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721443 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721382 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert\") pod \"1ac74618-fe58-430c-9d1e-5ffefee69bde\" (UID: \"1ac74618-fe58-430c-9d1e-5ffefee69bde\") " Jul 15 20:06:14.721563 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721537 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:06:14.721693 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721664 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:06:14.721757 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721685 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.721906 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721883 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:06:14.721946 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.721918 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config" (OuterVolumeSpecName: "console-config") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:06:14.723497 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.723465 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:06:14.723728 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.723700 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l" (OuterVolumeSpecName: "kube-api-access-lbg9l") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "kube-api-access-lbg9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:06:14.723842 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.723801 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ac74618-fe58-430c-9d1e-5ffefee69bde" (UID: "1ac74618-fe58-430c-9d1e-5ffefee69bde"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:06:14.822695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822655 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-oauth-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.822695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822688 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-oauth-serving-cert\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.822695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822697 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-serving-cert\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.822695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822707 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbg9l\" (UniqueName: \"kubernetes.io/projected/1ac74618-fe58-430c-9d1e-5ffefee69bde-kube-api-access-lbg9l\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.822960 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822717 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-console-config\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:14.822960 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:14.822725 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac74618-fe58-430c-9d1e-5ffefee69bde-trusted-ca-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:15.406351 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.406309 2557 generic.go:356] "Generic (PLEG): container finished" podID="1ac74618-fe58-430c-9d1e-5ffefee69bde" containerID="750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7" exitCode=0 Jul 15 20:06:15.406524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.406359 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf95f7756-7xdf4" Jul 15 20:06:15.406524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.406361 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf95f7756-7xdf4" event={"ID":"1ac74618-fe58-430c-9d1e-5ffefee69bde","Type":"ContainerDied","Data":"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7"} Jul 15 20:06:15.406524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.406411 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf95f7756-7xdf4" event={"ID":"1ac74618-fe58-430c-9d1e-5ffefee69bde","Type":"ContainerDied","Data":"63236ab7080bf84a1e4babae539327e8fb2d9bc8bff33cfffcb366ea830ec1c7"} Jul 15 20:06:15.406524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.406438 2557 scope.go:117] "RemoveContainer" containerID="750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7" Jul 15 20:06:15.415901 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.415883 2557 scope.go:117] "RemoveContainer" containerID="750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7" Jul 15 20:06:15.416186 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:06:15.416164 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7\": container with ID starting with 750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7 not found: ID does not exist" containerID="750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7" Jul 15 20:06:15.416313 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.416200 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7"} err="failed to get container status \"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7\": rpc error: code = NotFound desc = could not find container \"750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7\": container with ID starting with 750a170f97f9515a5310ff0583cbc56bce0a3362e4027c915c3367f3f0cd58f7 not found: ID does not exist" Jul 15 20:06:15.431122 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.431084 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:06:15.434568 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:15.434539 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bf95f7756-7xdf4"] Jul 15 20:06:16.121973 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:16.121926 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac74618-fe58-430c-9d1e-5ffefee69bde" path="/var/lib/kubelet/pods/1ac74618-fe58-430c-9d1e-5ffefee69bde/volumes" Jul 15 20:06:47.489797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.489757 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz"] Jul 15 20:06:47.490319 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.490116 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac74618-fe58-430c-9d1e-5ffefee69bde" containerName="console" Jul 15 20:06:47.490319 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.490127 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac74618-fe58-430c-9d1e-5ffefee69bde" containerName="console" Jul 15 20:06:47.490319 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.490195 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac74618-fe58-430c-9d1e-5ffefee69bde" containerName="console" Jul 15 20:06:47.494680 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.494660 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.498230 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.498197 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 20:06:47.498230 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.498208 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 20:06:47.498407 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.498255 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fhct9\"" Jul 15 20:06:47.510292 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.510265 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz"] Jul 15 20:06:47.614665 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.614624 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.614838 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.614683 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.614838 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.614780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfp7n\" (UniqueName: \"kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.715766 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.715718 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.715766 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.715776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfp7n\" (UniqueName: \"kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.716021 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.715821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.716142 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.716124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.716183 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.716158 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.725633 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.725607 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfp7n\" (UniqueName: \"kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.803774 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.803741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:47.882836 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.882806 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg"] Jul 15 20:06:47.886423 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.886400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:47.901631 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.901548 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg"] Jul 15 20:06:47.946204 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:47.946178 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz"] Jul 15 20:06:47.948457 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:06:47.948429 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c136809_eac3_488a_b6a7_30b3153ac1ee.slice/crio-4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008 WatchSource:0}: Error finding container 4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008: Status 404 returned error can't find the container with id 4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008 Jul 15 20:06:48.019383 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.019357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.019530 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.019390 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hq9q\" (UniqueName: \"kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.019596 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.019560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.120264 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.120230 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.120457 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.120298 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.120457 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.120323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hq9q\" (UniqueName: \"kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.120611 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.120588 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.120712 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.120688 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.130423 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.130395 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hq9q\" (UniqueName: \"kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.199162 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.199121 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:48.301419 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.301387 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88"] Jul 15 20:06:48.305279 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.305254 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.317365 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.317334 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88"] Jul 15 20:06:48.343055 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.343029 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg"] Jul 15 20:06:48.344250 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:06:48.344227 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd89395_ec78_4cc1_9107_2220645641b6.slice/crio-eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228 WatchSource:0}: Error finding container eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228: Status 404 returned error can't find the container with id eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228 Jul 15 20:06:48.422322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.422278 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.422474 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.422389 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7bkg\" (UniqueName: \"kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.422474 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.422426 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.523348 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.523310 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7bkg\" (UniqueName: \"kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.523348 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.523354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.523767 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.523423 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.523862 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.523841 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.523862 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.523855 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.533621 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.533598 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7bkg\" (UniqueName: \"kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.569493 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.569407 2557 generic.go:356] "Generic (PLEG): container finished" podID="fcd89395-ec78-4cc1-9107-2220645641b6" containerID="bdff4f8637d54654a865b73839ad83fbff28b6d1f846eb28ab9949d24e824f9e" exitCode=0 Jul 15 20:06:48.569645 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.569498 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" event={"ID":"fcd89395-ec78-4cc1-9107-2220645641b6","Type":"ContainerDied","Data":"bdff4f8637d54654a865b73839ad83fbff28b6d1f846eb28ab9949d24e824f9e"} Jul 15 20:06:48.569645 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.569567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" event={"ID":"fcd89395-ec78-4cc1-9107-2220645641b6","Type":"ContainerStarted","Data":"eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228"} Jul 15 20:06:48.571024 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.570997 2557 generic.go:356] "Generic (PLEG): container finished" podID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerID="4aa77473f4e456fb6947747f699f03a135c9b052558f10b99bbef2d5c4d98089" exitCode=0 Jul 15 20:06:48.571124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.571044 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" event={"ID":"0c136809-eac3-488a-b6a7-30b3153ac1ee","Type":"ContainerDied","Data":"4aa77473f4e456fb6947747f699f03a135c9b052558f10b99bbef2d5c4d98089"} Jul 15 20:06:48.571124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.571109 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" event={"ID":"0c136809-eac3-488a-b6a7-30b3153ac1ee","Type":"ContainerStarted","Data":"4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008"} Jul 15 20:06:48.616879 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.616845 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:48.678668 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.678638 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2"] Jul 15 20:06:48.682424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.682399 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.697510 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.697463 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2"] Jul 15 20:06:48.768599 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.768575 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88"] Jul 15 20:06:48.770458 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:06:48.770428 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ded2fff_f523_4165_beeb_2d73b4355624.slice/crio-1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd WatchSource:0}: Error finding container 1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd: Status 404 returned error can't find the container with id 1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd Jul 15 20:06:48.828271 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.828243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9qng\" (UniqueName: \"kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.828434 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.828283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.828434 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.828384 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.929322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.929280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9qng\" (UniqueName: \"kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.929322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.929326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.929541 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.929376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.929734 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.929719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.929780 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.929759 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.942631 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.942598 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9qng\" (UniqueName: \"kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:48.993620 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:48.993572 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:49.134528 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.134446 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2"] Jul 15 20:06:49.176959 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:06:49.176924 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5233b00a_b63d_4f1e_a83f_77d0d3d77e12.slice/crio-f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31 WatchSource:0}: Error finding container f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31: Status 404 returned error can't find the container with id f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31 Jul 15 20:06:49.577772 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.577736 2557 generic.go:356] "Generic (PLEG): container finished" podID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerID="19d391df34de232f8b2492e6029558aab8171e6f770143fc5d680a8ef44e0e5d" exitCode=0 Jul 15 20:06:49.578200 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.577781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" event={"ID":"5233b00a-b63d-4f1e-a83f-77d0d3d77e12","Type":"ContainerDied","Data":"19d391df34de232f8b2492e6029558aab8171e6f770143fc5d680a8ef44e0e5d"} Jul 15 20:06:49.578200 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.577831 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" event={"ID":"5233b00a-b63d-4f1e-a83f-77d0d3d77e12","Type":"ContainerStarted","Data":"f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31"} Jul 15 20:06:49.579865 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.579839 2557 generic.go:356] "Generic (PLEG): container finished" podID="fcd89395-ec78-4cc1-9107-2220645641b6" containerID="7d76f504fa7f821811f454a43677b4a870233f5a313249a7d140b614932be201" exitCode=0 Jul 15 20:06:49.579977 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.579921 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" event={"ID":"fcd89395-ec78-4cc1-9107-2220645641b6","Type":"ContainerDied","Data":"7d76f504fa7f821811f454a43677b4a870233f5a313249a7d140b614932be201"} Jul 15 20:06:49.581495 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.581475 2557 generic.go:356] "Generic (PLEG): container finished" podID="3ded2fff-f523-4165-beeb-2d73b4355624" containerID="3ba20359904e065de2d78684ffba125e244e09471ac99bea929a0296eb6961d4" exitCode=0 Jul 15 20:06:49.581562 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.581524 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerDied","Data":"3ba20359904e065de2d78684ffba125e244e09471ac99bea929a0296eb6961d4"} Jul 15 20:06:49.581616 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.581573 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerStarted","Data":"1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd"} Jul 15 20:06:49.583506 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.583480 2557 generic.go:356] "Generic (PLEG): container finished" podID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerID="e0994d78cd5c3551b4c0df66b646705c3751150bbd49afe4daa5f1df520fbfaa" exitCode=0 Jul 15 20:06:49.583614 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:49.583545 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" event={"ID":"0c136809-eac3-488a-b6a7-30b3153ac1ee","Type":"ContainerDied","Data":"e0994d78cd5c3551b4c0df66b646705c3751150bbd49afe4daa5f1df520fbfaa"} Jul 15 20:06:50.590171 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.590056 2557 generic.go:356] "Generic (PLEG): container finished" podID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerID="656157381e095bca7291178981f6a9ae4fe1c5df11e45295102cd4cabd38a111" exitCode=0 Jul 15 20:06:50.590171 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.590102 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" event={"ID":"5233b00a-b63d-4f1e-a83f-77d0d3d77e12","Type":"ContainerDied","Data":"656157381e095bca7291178981f6a9ae4fe1c5df11e45295102cd4cabd38a111"} Jul 15 20:06:50.592433 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.592403 2557 generic.go:356] "Generic (PLEG): container finished" podID="fcd89395-ec78-4cc1-9107-2220645641b6" containerID="1d1d5925baee066d2144f46f91eb3a36b8856cee46b07f58eba3c169cf811de7" exitCode=0 Jul 15 20:06:50.592542 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.592436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" event={"ID":"fcd89395-ec78-4cc1-9107-2220645641b6","Type":"ContainerDied","Data":"1d1d5925baee066d2144f46f91eb3a36b8856cee46b07f58eba3c169cf811de7"} Jul 15 20:06:50.594480 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.594450 2557 generic.go:356] "Generic (PLEG): container finished" podID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerID="8ad4fbe3c1462934cf1348f1673b5bdbdce6c838eb3e424f6256aaf382e4a34a" exitCode=0 Jul 15 20:06:50.594572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:50.594531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" event={"ID":"0c136809-eac3-488a-b6a7-30b3153ac1ee","Type":"ContainerDied","Data":"8ad4fbe3c1462934cf1348f1673b5bdbdce6c838eb3e424f6256aaf382e4a34a"} Jul 15 20:06:51.602530 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.602486 2557 generic.go:356] "Generic (PLEG): container finished" podID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerID="0693f9f7f745effe90c6949f1fd0cfdd9ebfb2001a15ace27c3ca4e05ade47f2" exitCode=0 Jul 15 20:06:51.602936 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.602578 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" event={"ID":"5233b00a-b63d-4f1e-a83f-77d0d3d77e12","Type":"ContainerDied","Data":"0693f9f7f745effe90c6949f1fd0cfdd9ebfb2001a15ace27c3ca4e05ade47f2"} Jul 15 20:06:51.604518 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.604491 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerStarted","Data":"0e371ef454905ac31d7736c652708807cc326afe14b706ce924b5dd6b3c7448a"} Jul 15 20:06:51.759371 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.759336 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:51.762226 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.762202 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:51.767230 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767206 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util\") pod \"0c136809-eac3-488a-b6a7-30b3153ac1ee\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " Jul 15 20:06:51.767356 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767298 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hq9q\" (UniqueName: \"kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q\") pod \"fcd89395-ec78-4cc1-9107-2220645641b6\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " Jul 15 20:06:51.767356 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767344 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfp7n\" (UniqueName: \"kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n\") pod \"0c136809-eac3-488a-b6a7-30b3153ac1ee\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " Jul 15 20:06:51.767460 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767388 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util\") pod \"fcd89395-ec78-4cc1-9107-2220645641b6\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " Jul 15 20:06:51.767460 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767415 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle\") pod \"fcd89395-ec78-4cc1-9107-2220645641b6\" (UID: \"fcd89395-ec78-4cc1-9107-2220645641b6\") " Jul 15 20:06:51.767460 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767449 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle\") pod \"0c136809-eac3-488a-b6a7-30b3153ac1ee\" (UID: \"0c136809-eac3-488a-b6a7-30b3153ac1ee\") " Jul 15 20:06:51.767975 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.767946 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle" (OuterVolumeSpecName: "bundle") pod "fcd89395-ec78-4cc1-9107-2220645641b6" (UID: "fcd89395-ec78-4cc1-9107-2220645641b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:51.768229 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.768207 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle" (OuterVolumeSpecName: "bundle") pod "0c136809-eac3-488a-b6a7-30b3153ac1ee" (UID: "0c136809-eac3-488a-b6a7-30b3153ac1ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:51.770228 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.770161 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q" (OuterVolumeSpecName: "kube-api-access-6hq9q") pod "fcd89395-ec78-4cc1-9107-2220645641b6" (UID: "fcd89395-ec78-4cc1-9107-2220645641b6"). InnerVolumeSpecName "kube-api-access-6hq9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:06:51.770228 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.770196 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n" (OuterVolumeSpecName: "kube-api-access-cfp7n") pod "0c136809-eac3-488a-b6a7-30b3153ac1ee" (UID: "0c136809-eac3-488a-b6a7-30b3153ac1ee"). InnerVolumeSpecName "kube-api-access-cfp7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:06:51.772737 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.772718 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util" (OuterVolumeSpecName: "util") pod "0c136809-eac3-488a-b6a7-30b3153ac1ee" (UID: "0c136809-eac3-488a-b6a7-30b3153ac1ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:51.773040 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.773022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util" (OuterVolumeSpecName: "util") pod "fcd89395-ec78-4cc1-9107-2220645641b6" (UID: "fcd89395-ec78-4cc1-9107-2220645641b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:51.869129 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869061 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hq9q\" (UniqueName: \"kubernetes.io/projected/fcd89395-ec78-4cc1-9107-2220645641b6-kube-api-access-6hq9q\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:51.869129 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869124 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfp7n\" (UniqueName: \"kubernetes.io/projected/0c136809-eac3-488a-b6a7-30b3153ac1ee-kube-api-access-cfp7n\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:51.869129 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869135 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:51.869368 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869144 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcd89395-ec78-4cc1-9107-2220645641b6-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:51.869368 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869153 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:51.869368 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:51.869162 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c136809-eac3-488a-b6a7-30b3153ac1ee-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:52.611817 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.611787 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" Jul 15 20:06:52.612252 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.611785 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg" event={"ID":"fcd89395-ec78-4cc1-9107-2220645641b6","Type":"ContainerDied","Data":"eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228"} Jul 15 20:06:52.612252 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.611924 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb499841d98e5c111408df9afd5e258099358fbc34c9bb210c7a9f5134630228" Jul 15 20:06:52.613631 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.613604 2557 generic.go:356] "Generic (PLEG): container finished" podID="3ded2fff-f523-4165-beeb-2d73b4355624" containerID="0e371ef454905ac31d7736c652708807cc326afe14b706ce924b5dd6b3c7448a" exitCode=0 Jul 15 20:06:52.613773 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.613690 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerDied","Data":"0e371ef454905ac31d7736c652708807cc326afe14b706ce924b5dd6b3c7448a"} Jul 15 20:06:52.615755 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.615730 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" event={"ID":"0c136809-eac3-488a-b6a7-30b3153ac1ee","Type":"ContainerDied","Data":"4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008"} Jul 15 20:06:52.615817 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.615762 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz" Jul 15 20:06:52.615817 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.615776 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e1970c3246166263d4eecd773572038be5dd97405af79eb8017a49526e90008" Jul 15 20:06:52.753560 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.753530 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:52.776301 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.776270 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9qng\" (UniqueName: \"kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng\") pod \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " Jul 15 20:06:52.776463 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.776318 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle\") pod \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " Jul 15 20:06:52.776463 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.776413 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util\") pod \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\" (UID: \"5233b00a-b63d-4f1e-a83f-77d0d3d77e12\") " Jul 15 20:06:52.776820 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.776788 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle" (OuterVolumeSpecName: "bundle") pod "5233b00a-b63d-4f1e-a83f-77d0d3d77e12" (UID: "5233b00a-b63d-4f1e-a83f-77d0d3d77e12"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:52.778601 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.778567 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng" (OuterVolumeSpecName: "kube-api-access-z9qng") pod "5233b00a-b63d-4f1e-a83f-77d0d3d77e12" (UID: "5233b00a-b63d-4f1e-a83f-77d0d3d77e12"). InnerVolumeSpecName "kube-api-access-z9qng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:06:52.781789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.781758 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util" (OuterVolumeSpecName: "util") pod "5233b00a-b63d-4f1e-a83f-77d0d3d77e12" (UID: "5233b00a-b63d-4f1e-a83f-77d0d3d77e12"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:52.877022 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.876934 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:52.877022 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.876969 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9qng\" (UniqueName: \"kubernetes.io/projected/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-kube-api-access-z9qng\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:52.877022 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:52.876980 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5233b00a-b63d-4f1e-a83f-77d0d3d77e12-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:53.622846 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:53.622810 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" Jul 15 20:06:53.622846 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:53.622817 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2" event={"ID":"5233b00a-b63d-4f1e-a83f-77d0d3d77e12","Type":"ContainerDied","Data":"f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31"} Jul 15 20:06:53.623356 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:53.622896 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f54aa2955475f72077d9fd914fc6f1f9b10cdb72395cb9822337e449cfa83f31" Jul 15 20:06:53.625004 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:53.624980 2557 generic.go:356] "Generic (PLEG): container finished" podID="3ded2fff-f523-4165-beeb-2d73b4355624" containerID="f5ec36c8de10f8499c71bbee29b1395b5d55a12815c4f8aea5df4ac1e987f642" exitCode=0 Jul 15 20:06:53.625160 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:53.625028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerDied","Data":"f5ec36c8de10f8499c71bbee29b1395b5d55a12815c4f8aea5df4ac1e987f642"} Jul 15 20:06:54.762609 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.762578 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:54.793898 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.793869 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util\") pod \"3ded2fff-f523-4165-beeb-2d73b4355624\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " Jul 15 20:06:54.794054 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.793913 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7bkg\" (UniqueName: \"kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg\") pod \"3ded2fff-f523-4165-beeb-2d73b4355624\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " Jul 15 20:06:54.794054 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.794015 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle\") pod \"3ded2fff-f523-4165-beeb-2d73b4355624\" (UID: \"3ded2fff-f523-4165-beeb-2d73b4355624\") " Jul 15 20:06:54.794554 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.794526 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle" (OuterVolumeSpecName: "bundle") pod "3ded2fff-f523-4165-beeb-2d73b4355624" (UID: "3ded2fff-f523-4165-beeb-2d73b4355624"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:54.796295 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.796268 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg" (OuterVolumeSpecName: "kube-api-access-m7bkg") pod "3ded2fff-f523-4165-beeb-2d73b4355624" (UID: "3ded2fff-f523-4165-beeb-2d73b4355624"). InnerVolumeSpecName "kube-api-access-m7bkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:06:54.798782 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.798757 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util" (OuterVolumeSpecName: "util") pod "3ded2fff-f523-4165-beeb-2d73b4355624" (UID: "3ded2fff-f523-4165-beeb-2d73b4355624"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:06:54.895130 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.895003 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-util\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:54.895130 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.895043 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7bkg\" (UniqueName: \"kubernetes.io/projected/3ded2fff-f523-4165-beeb-2d73b4355624-kube-api-access-m7bkg\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:54.895130 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:54.895059 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ded2fff-f523-4165-beeb-2d73b4355624-bundle\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:06:55.637926 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:55.637890 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" event={"ID":"3ded2fff-f523-4165-beeb-2d73b4355624","Type":"ContainerDied","Data":"1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd"} Jul 15 20:06:55.637926 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:55.637932 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88" Jul 15 20:06:55.638165 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:06:55.637944 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cb0c1659643f9d14ffab9d1bc55552bb76984f9627c1b257aebad5b361479bd" Jul 15 20:08:17.349798 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.349756 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:08:17.350322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350272 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="pull" Jul 15 20:08:17.350322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350292 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="pull" Jul 15 20:08:17.350322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350307 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="util" Jul 15 20:08:17.350322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350316 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350327 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350336 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350353 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350362 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350374 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350381 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350391 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="pull" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350398 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="pull" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350409 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350417 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350427 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="pull" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350435 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="pull" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350443 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350453 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="extract" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350466 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350474 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350485 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350492 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="util" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350510 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="pull" Jul 15 20:08:17.350535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350518 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="pull" Jul 15 20:08:17.351783 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350603 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c136809-eac3-488a-b6a7-30b3153ac1ee" containerName="extract" Jul 15 20:08:17.351783 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350618 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd89395-ec78-4cc1-9107-2220645641b6" containerName="extract" Jul 15 20:08:17.351783 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350628 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5233b00a-b63d-4f1e-a83f-77d0d3d77e12" containerName="extract" Jul 15 20:08:17.351783 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.350637 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ded2fff-f523-4165-beeb-2d73b4355624" containerName="extract" Jul 15 20:08:17.353973 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.353954 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.358822 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.358804 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fhct9\"" Jul 15 20:08:17.361553 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.361530 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 20:08:17.361653 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.361624 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 20:08:17.362980 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.362964 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 20:08:17.449346 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.449315 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:08:17.499759 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.499725 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:08:17.528691 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.528651 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmgw7\" (UniqueName: \"kubernetes.io/projected/7632d12b-fa25-4336-9b90-0081e7319920-kube-api-access-hmgw7\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.528868 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.528743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7632d12b-fa25-4336-9b90-0081e7319920-config-file\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.629610 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.629518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7632d12b-fa25-4336-9b90-0081e7319920-config-file\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.629610 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.629568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmgw7\" (UniqueName: \"kubernetes.io/projected/7632d12b-fa25-4336-9b90-0081e7319920-kube-api-access-hmgw7\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.630191 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.630171 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7632d12b-fa25-4336-9b90-0081e7319920-config-file\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.639729 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.639704 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmgw7\" (UniqueName: \"kubernetes.io/projected/7632d12b-fa25-4336-9b90-0081e7319920-kube-api-access-hmgw7\") pod \"limitador-limitador-54d6d988d5-zm2zh\" (UID: \"7632d12b-fa25-4336-9b90-0081e7319920\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.663677 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.663636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:17.804852 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:17.804821 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:08:17.806251 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:08:17.806213 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7632d12b_fa25_4336_9b90_0081e7319920.slice/crio-fb9a7e23ed0272258b70192ac8c7cc711bdf803c51bc7778ce0b3f3918429e6c WatchSource:0}: Error finding container fb9a7e23ed0272258b70192ac8c7cc711bdf803c51bc7778ce0b3f3918429e6c: Status 404 returned error can't find the container with id fb9a7e23ed0272258b70192ac8c7cc711bdf803c51bc7778ce0b3f3918429e6c Jul 15 20:08:18.060757 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:18.060724 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" event={"ID":"7632d12b-fa25-4336-9b90-0081e7319920","Type":"ContainerStarted","Data":"fb9a7e23ed0272258b70192ac8c7cc711bdf803c51bc7778ce0b3f3918429e6c"} Jul 15 20:08:21.078811 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:21.078775 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" event={"ID":"7632d12b-fa25-4336-9b90-0081e7319920","Type":"ContainerStarted","Data":"5d43ad7edb8b5bcf83686a041e63f990f2261ee75fa1f16b8989b72e8b63b43c"} Jul 15 20:08:21.079207 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:21.078864 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:08:21.102877 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:21.102822 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" podStartSLOduration=1.3972680149999999 podStartE2EDuration="4.102805148s" totalImagesPullingTime="2.705537133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:08:21.100880805 +0000 UTC m=+593.972874237" watchObservedRunningTime="2026-07-15 20:08:21.102805148 +0000 UTC m=+593.974798579" Jul 15 20:08:27.996047 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:27.996017 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:08:27.999624 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:27.999600 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:08:28.004679 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:28.004653 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:08:28.008137 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:28.008119 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:08:32.085191 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:08:32.085159 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zm2zh" Jul 15 20:09:45.741199 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:09:45.741160 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:10:15.046209 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:15.046173 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:10:28.849399 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:28.849296 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:10:39.553762 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.553728 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd"] Jul 15 20:10:39.556140 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.556123 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.560354 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.560331 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rkl7w\"" Jul 15 20:10:39.560495 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.560361 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 20:10:39.560495 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.560334 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 20:10:39.560495 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.560334 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 20:10:39.569941 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.569914 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd"] Jul 15 20:10:39.699166 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.699166 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699167 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.699437 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.699437 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699220 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.699437 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjwcs\" (UniqueName: \"kubernetes.io/projected/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-kube-api-access-rjwcs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.699437 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.699317 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.800815 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.800783 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.800815 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.800816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.800835 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjwcs\" (UniqueName: \"kubernetes.io/projected/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-kube-api-access-rjwcs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.800972 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801198 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.801147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801198 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.801175 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801357 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.801337 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801418 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.801342 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.801548 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.801528 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.803414 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.803389 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.803684 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.803664 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.809594 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.809532 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjwcs\" (UniqueName: \"kubernetes.io/projected/e968bcf0-aa3a-4a31-8a1c-87ab36ff4362-kube-api-access-rjwcs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqxmd\" (UID: \"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:39.866564 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:39.866529 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:40.002612 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:40.002582 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd"] Jul 15 20:10:40.004554 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:10:40.004524 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode968bcf0_aa3a_4a31_8a1c_87ab36ff4362.slice/crio-1ddb800a4aacba80a19fcd95a5b278d0a3845a1a41d44be7165e0f31b59f856d WatchSource:0}: Error finding container 1ddb800a4aacba80a19fcd95a5b278d0a3845a1a41d44be7165e0f31b59f856d: Status 404 returned error can't find the container with id 1ddb800a4aacba80a19fcd95a5b278d0a3845a1a41d44be7165e0f31b59f856d Jul 15 20:10:40.005839 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:40.005821 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:10:40.818522 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:40.818482 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" event={"ID":"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362","Type":"ContainerStarted","Data":"1ddb800a4aacba80a19fcd95a5b278d0a3845a1a41d44be7165e0f31b59f856d"} Jul 15 20:10:40.848700 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:40.848667 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:10:41.824432 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:41.824392 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" event={"ID":"e968bcf0-aa3a-4a31-8a1c-87ab36ff4362","Type":"ContainerStarted","Data":"c070e4ec10981fe502dd33e9d3776d4951a36c453996c509f117c4570b2ef102"} Jul 15 20:10:41.846672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:41.846621 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" podStartSLOduration=1.85176102 podStartE2EDuration="2.846604997s" totalImagesPullingTime="994.843977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:10:41.845005232 +0000 UTC m=+734.716998678" watchObservedRunningTime="2026-07-15 20:10:41.846604997 +0000 UTC m=+734.718598429" Jul 15 20:10:49.553320 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:49.553277 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:10:49.867225 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:49.867135 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:49.867372 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:49.867245 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:49.879778 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:49.879755 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:50.881316 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:50.881283 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqxmd" Jul 15 20:10:55.543771 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:10:55.543733 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:11:05.752264 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:11:05.752229 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:12:47.084118 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:12:47.084065 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:12:59.442649 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:12:59.442614 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:13:09.549360 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:09.549321 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:13:18.063280 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:18.063246 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:13:28.032524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:28.032447 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:13:28.042947 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:28.042916 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:13:28.043760 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:28.043730 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:13:28.047632 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:28.047608 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:13:28.102308 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:13:28.102278 2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]" Jul 15 20:13:29.050710 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:29.050673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:13:55.169396 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:13:55.169359 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:14:10.178303 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:14:10.178266 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:14:48.161316 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:14:48.161277 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:14:57.751667 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:14:57.751581 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:15:00.168666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.168628 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7"] Jul 15 20:15:00.171965 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.171946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" Jul 15 20:15:00.174860 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.174839 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:15:00.181373 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.181355 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" Jul 15 20:15:00.190265 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.190236 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7"] Jul 15 20:15:00.328405 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:00.328372 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7"] Jul 15 20:15:00.330034 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:15:00.330005 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd704b47_363c_4850_823d_626ad2643c47.slice/crio-632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b WatchSource:0}: Error finding container 632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b: Status 404 returned error can't find the container with id 632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b Jul 15 20:15:01.155447 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:01.155409 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" event={"ID":"dd704b47-363c-4850-823d-626ad2643c47","Type":"ContainerStarted","Data":"632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b"} Jul 15 20:15:03.169017 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:03.168907 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" event={"ID":"dd704b47-363c-4850-823d-626ad2643c47","Type":"ContainerStarted","Data":"979d4dd1fc3c5cc9665fd73adc28d71826907490cbcc956f12ea58275f70b9ac"} Jul 15 20:15:03.186698 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:03.186643 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" podStartSLOduration=0.926943459 podStartE2EDuration="3.186627687s" totalImagesPullingTime="2.259684228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:15:03.184703625 +0000 UTC m=+996.056697056" watchObservedRunningTime="2026-07-15 20:15:03.186627687 +0000 UTC m=+996.058621119" Jul 15 20:15:04.176009 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:04.175977 2557 generic.go:356] "Generic (PLEG): container finished" podID="dd704b47-363c-4850-823d-626ad2643c47" containerID="979d4dd1fc3c5cc9665fd73adc28d71826907490cbcc956f12ea58275f70b9ac" exitCode=0 Jul 15 20:15:04.176377 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:04.176089 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" event={"ID":"dd704b47-363c-4850-823d-626ad2643c47","Type":"ContainerDied","Data":"979d4dd1fc3c5cc9665fd73adc28d71826907490cbcc956f12ea58275f70b9ac"} Jul 15 20:15:05.316653 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:05.316627 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" Jul 15 20:15:06.187850 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:06.187823 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" Jul 15 20:15:06.188025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:06.187822 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-qx4s7" event={"ID":"dd704b47-363c-4850-823d-626ad2643c47","Type":"ContainerDied","Data":"632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b"} Jul 15 20:15:06.188025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:06.187938 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="632eed292af17e845ddea178c343282e099e2704f69231d9be7adbc93029919b" Jul 15 20:15:07.553224 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:07.553184 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:15:23.647048 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:23.647012 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:15:34.968585 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:34.968549 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:15:44.589524 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:44.589482 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:15:57.548037 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:15:57.547996 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:16:08.156794 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:16:08.156753 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:16:29.070773 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:16:29.070689 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:16:40.748408 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:16:40.748369 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:17:06.352688 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:17:06.352641 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:17:18.677586 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:17:18.677543 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:17:29.757232 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:17:29.757182 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:17:39.469702 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:17:39.469663 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:17:50.043867 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:17:50.043790 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:18:00.044042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:00.044001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:18:09.450552 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:09.450509 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:18:28.080360 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:28.080330 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:18:28.082169 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:28.082144 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:18:28.084249 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:28.084229 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:18:28.086058 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:18:28.086035 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:19:55.852732 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:19:55.852695 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:06.950988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:06.950950 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:16.386126 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:16.386084 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:30.248806 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:30.248766 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:40.253435 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:40.253396 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:49.945119 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:49.944998 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:20:59.145179 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:20:59.145142 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:21:10.746236 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:21:10.746200 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:21:20.151916 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:21:20.151877 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:21:28.247387 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:21:28.247345 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:21:37.773752 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:21:37.773703 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:21:44.542925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:21:44.542875 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:22:06.741678 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:22:06.741640 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:22:16.645306 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:22:16.645267 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:22:26.549623 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:22:26.549539 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:22:37.653921 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:22:37.648598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:22:54.258378 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:22:54.258336 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:23:03.947127 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:03.947067 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:23:14.103800 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:14.103762 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:23:25.648084 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:25.648045 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:23:28.125032 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:28.124998 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:23:28.125504 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:28.125304 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:23:28.128613 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:28.128593 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:23:28.128879 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:28.128864 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:23:40.178468 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:40.178435 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:23:48.252573 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:23:48.252536 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:24:03.459486 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:24:03.459368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:24:15.957628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:24:15.957577 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:24:26.356891 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:24:26.356853 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:24:39.763968 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:24:39.763932 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:24:47.856555 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:24:47.856516 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:00.256502 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:00.256463 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:13.151862 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:13.151824 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:25.345496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:25.345392 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:37.754151 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:37.754111 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:47.251982 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:47.251940 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:25:57.277965 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:25:57.277931 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:26:04.702172 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.702130 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:04.702661 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.702639 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd704b47-363c-4850-823d-626ad2643c47" containerName="cleanup" Jul 15 20:26:04.702746 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.702665 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd704b47-363c-4850-823d-626ad2643c47" containerName="cleanup" Jul 15 20:26:04.702806 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.702795 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd704b47-363c-4850-823d-626ad2643c47" containerName="cleanup" Jul 15 20:26:04.705600 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.705577 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:04.708970 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.708948 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:26:04.709122 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.708961 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:26:04.725822 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.725794 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:04.825525 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.825480 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd\" (UID: \"a7db2b82-838b-46f0-ad38-605ec152f392\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:04.926967 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.926927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd\" (UID: \"a7db2b82-838b-46f0-ad38-605ec152f392\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:04.927641 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:04.927612 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd\" (UID: \"a7db2b82-838b-46f0-ad38-605ec152f392\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:05.016308 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:05.016218 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:05.164512 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:05.164489 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:05.166708 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:26:05.166682 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7db2b82_838b_46f0_ad38_605ec152f392.slice/crio-3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5 WatchSource:0}: Error finding container 3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5: Status 404 returned error can't find the container with id 3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5 Jul 15 20:26:05.684378 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:05.684276 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerStarted","Data":"a13803ab3c89774056794874a408b4620a593a8ac44b5b179bc55b61be6168d9"} Jul 15 20:26:05.684378 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:05.684341 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerStarted","Data":"3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5"} Jul 15 20:26:05.706022 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:05.705962 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" podStartSLOduration=1.705945508 podStartE2EDuration="1.705945508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:26:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:26:05.704313633 +0000 UTC m=+1658.576307066" watchObservedRunningTime="2026-07-15 20:26:05.705945508 +0000 UTC m=+1658.577938939" Jul 15 20:26:06.691392 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:06.691355 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7db2b82-838b-46f0-ad38-605ec152f392" containerID="a13803ab3c89774056794874a408b4620a593a8ac44b5b179bc55b61be6168d9" exitCode=7 Jul 15 20:26:06.691602 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:06.691435 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerDied","Data":"a13803ab3c89774056794874a408b4620a593a8ac44b5b179bc55b61be6168d9"} Jul 15 20:26:06.691698 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:06.691680 2557 scope.go:117] "RemoveContainer" containerID="a13803ab3c89774056794874a408b4620a593a8ac44b5b179bc55b61be6168d9" Jul 15 20:26:07.698193 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:07.698103 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7db2b82-838b-46f0-ad38-605ec152f392" containerID="0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b" exitCode=7 Jul 15 20:26:07.698193 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:07.698163 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerDied","Data":"0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b"} Jul 15 20:26:07.698610 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:07.698222 2557 scope.go:117] "RemoveContainer" containerID="a13803ab3c89774056794874a408b4620a593a8ac44b5b179bc55b61be6168d9" Jul 15 20:26:07.698610 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:07.698413 2557 scope.go:117] "RemoveContainer" containerID="0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b" Jul 15 20:26:07.698686 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:26:07.698618 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd_odh-ai-gateway-infra(a7db2b82-838b-46f0-ad38-605ec152f392)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" Jul 15 20:26:08.705633 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:08.705543 2557 scope.go:117] "RemoveContainer" containerID="0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b" Jul 15 20:26:08.705969 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:26:08.705728 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd_odh-ai-gateway-infra(a7db2b82-838b-46f0-ad38-605ec152f392)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" Jul 15 20:26:11.154777 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:11.154737 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:26:21.116109 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:21.116052 2557 scope.go:117] "RemoveContainer" containerID="0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b" Jul 15 20:26:21.775997 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:21.775905 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerStarted","Data":"24f27202e27f7a6db9ad4662c87f2f829a3a35c6dd4037b447f870b79661cae2"} Jul 15 20:26:22.783970 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:22.783932 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7db2b82-838b-46f0-ad38-605ec152f392" containerID="24f27202e27f7a6db9ad4662c87f2f829a3a35c6dd4037b447f870b79661cae2" exitCode=0 Jul 15 20:26:22.784440 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:22.784004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerDied","Data":"24f27202e27f7a6db9ad4662c87f2f829a3a35c6dd4037b447f870b79661cae2"} Jul 15 20:26:22.784440 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:22.784066 2557 scope.go:117] "RemoveContainer" containerID="0c02d791e5d65c75dcc214b56a8938608a97191f8cc6c6ba4799ee83fdee587b" Jul 15 20:26:22.808067 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:22.808028 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:23.927313 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:23.927281 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:24.000656 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.000623 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca\") pod \"a7db2b82-838b-46f0-ad38-605ec152f392\" (UID: \"a7db2b82-838b-46f0-ad38-605ec152f392\") " Jul 15 20:26:24.000972 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.000946 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7db2b82-838b-46f0-ad38-605ec152f392" (UID: "a7db2b82-838b-46f0-ad38-605ec152f392"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:26:24.102004 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.101924 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7db2b82-838b-46f0-ad38-605ec152f392-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:26:24.797711 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.797685 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" Jul 15 20:26:24.797884 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.797681 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd" event={"ID":"a7db2b82-838b-46f0-ad38-605ec152f392","Type":"ContainerDied","Data":"3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5"} Jul 15 20:26:24.797884 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.797814 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e5c9046a02e33f5aad5cb84b86687cc2c2e3d3c49f45aa245f5bef3165462b5" Jul 15 20:26:24.834496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.834456 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:24.839347 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:24.839320 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-kw4vd"] Jul 15 20:26:26.121757 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:26:26.121719 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" path="/var/lib/kubelet/pods/a7db2b82-838b-46f0-ad38-605ec152f392/volumes" Jul 15 20:27:04.743287 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743193 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl"] Jul 15 20:27:04.743887 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743741 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.743887 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743762 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.743887 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743780 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.743887 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743789 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.744569 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743912 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.744569 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743928 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.744569 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.743937 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:04.747299 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.747277 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:04.751166 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.751144 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:27:04.751285 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.751248 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:27:04.759350 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.759324 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl"] Jul 15 20:27:04.872747 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.872685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl\" (UID: \"e582465d-544b-4157-966e-b612f16fbb5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:04.973473 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.973420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl\" (UID: \"e582465d-544b-4157-966e-b612f16fbb5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:04.974102 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:04.974052 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl\" (UID: \"e582465d-544b-4157-966e-b612f16fbb5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:05.058424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:05.058393 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:05.202970 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:05.202940 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl"] Jul 15 20:27:05.204429 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:27:05.204398 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode582465d_544b_4157_966e_b612f16fbb5e.slice/crio-107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd WatchSource:0}: Error finding container 107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd: Status 404 returned error can't find the container with id 107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd Jul 15 20:27:06.019262 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:06.019226 2557 generic.go:356] "Generic (PLEG): container finished" podID="e582465d-544b-4157-966e-b612f16fbb5e" containerID="15e1ca835ad4c5ee376440aebdf905e4c543eba06657224a62376dfe13b7d87c" exitCode=0 Jul 15 20:27:06.019646 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:06.019276 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" event={"ID":"e582465d-544b-4157-966e-b612f16fbb5e","Type":"ContainerDied","Data":"15e1ca835ad4c5ee376440aebdf905e4c543eba06657224a62376dfe13b7d87c"} Jul 15 20:27:06.019646 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:06.019323 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" event={"ID":"e582465d-544b-4157-966e-b612f16fbb5e","Type":"ContainerStarted","Data":"107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd"} Jul 15 20:27:07.156461 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:07.156436 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:07.297014 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:07.296979 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca\") pod \"e582465d-544b-4157-966e-b612f16fbb5e\" (UID: \"e582465d-544b-4157-966e-b612f16fbb5e\") " Jul 15 20:27:07.297389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:07.297364 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e582465d-544b-4157-966e-b612f16fbb5e" (UID: "e582465d-544b-4157-966e-b612f16fbb5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:27:07.398066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:07.398023 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e582465d-544b-4157-966e-b612f16fbb5e-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:27:08.031379 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:08.031338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" event={"ID":"e582465d-544b-4157-966e-b612f16fbb5e","Type":"ContainerDied","Data":"107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd"} Jul 15 20:27:08.031557 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:08.031395 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="107c6a188e9bcfe6dd08884b56ed97140bd27220236ce0ada0f0c6edd39521dd" Jul 15 20:27:08.031557 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:08.031361 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl" Jul 15 20:27:17.944996 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:17.944958 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:27:34.668916 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:34.668876 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:27:36.583960 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.583924 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:36.584413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.584313 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:36.584413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.584325 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7db2b82-838b-46f0-ad38-605ec152f392" containerName="revoke-keys" Jul 15 20:27:36.584413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.584336 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e582465d-544b-4157-966e-b612f16fbb5e" containerName="revoke-keys" Jul 15 20:27:36.584413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.584342 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e582465d-544b-4157-966e-b612f16fbb5e" containerName="revoke-keys" Jul 15 20:27:36.584413 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.584398 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e582465d-544b-4157-966e-b612f16fbb5e" containerName="revoke-keys" Jul 15 20:27:36.587496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.587479 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:36.590483 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.590465 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:27:36.590621 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.590604 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:27:36.603444 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.603418 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:36.650676 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.650644 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx\" (UID: \"45de28a1-7377-4733-a48c-31624d452f6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:36.752048 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.752000 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx\" (UID: \"45de28a1-7377-4733-a48c-31624d452f6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:36.752669 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.752649 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx\" (UID: \"45de28a1-7377-4733-a48c-31624d452f6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:36.897626 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:36.897539 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:37.039789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:37.039760 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:37.041811 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:27:37.041777 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45de28a1_7377_4733_a48c_31624d452f6a.slice/crio-d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045 WatchSource:0}: Error finding container d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045: Status 404 returned error can't find the container with id d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045 Jul 15 20:27:37.193482 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:37.193409 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" event={"ID":"45de28a1-7377-4733-a48c-31624d452f6a","Type":"ContainerStarted","Data":"d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045"} Jul 15 20:27:38.202220 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:38.202183 2557 generic.go:356] "Generic (PLEG): container finished" podID="45de28a1-7377-4733-a48c-31624d452f6a" containerID="1b8e91e75ed36c37811bfa9bbccbe8977e25fa3e6fdeb9c70029d804ba5aa4f7" exitCode=7 Jul 15 20:27:38.202672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:38.202259 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" event={"ID":"45de28a1-7377-4733-a48c-31624d452f6a","Type":"ContainerDied","Data":"1b8e91e75ed36c37811bfa9bbccbe8977e25fa3e6fdeb9c70029d804ba5aa4f7"} Jul 15 20:27:38.202672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:38.202497 2557 scope.go:117] "RemoveContainer" containerID="1b8e91e75ed36c37811bfa9bbccbe8977e25fa3e6fdeb9c70029d804ba5aa4f7" Jul 15 20:27:39.210517 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:39.210420 2557 generic.go:356] "Generic (PLEG): container finished" podID="45de28a1-7377-4733-a48c-31624d452f6a" containerID="5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997" exitCode=7 Jul 15 20:27:39.210517 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:39.210502 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" event={"ID":"45de28a1-7377-4733-a48c-31624d452f6a","Type":"ContainerDied","Data":"5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997"} Jul 15 20:27:39.211030 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:39.210559 2557 scope.go:117] "RemoveContainer" containerID="1b8e91e75ed36c37811bfa9bbccbe8977e25fa3e6fdeb9c70029d804ba5aa4f7" Jul 15 20:27:39.211030 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:39.210752 2557 scope.go:117] "RemoveContainer" containerID="5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997" Jul 15 20:27:39.211030 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:27:39.211019 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx_odh-ai-gateway-infra(45de28a1-7377-4733-a48c-31624d452f6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" podUID="45de28a1-7377-4733-a48c-31624d452f6a" Jul 15 20:27:40.217174 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:40.217122 2557 scope.go:117] "RemoveContainer" containerID="5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997" Jul 15 20:27:40.217556 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:27:40.217327 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx_odh-ai-gateway-infra(45de28a1-7377-4733-a48c-31624d452f6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" podUID="45de28a1-7377-4733-a48c-31624d452f6a" Jul 15 20:27:53.116687 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:53.116657 2557 scope.go:117] "RemoveContainer" containerID="5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997" Jul 15 20:27:54.293913 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:54.293878 2557 generic.go:356] "Generic (PLEG): container finished" podID="45de28a1-7377-4733-a48c-31624d452f6a" containerID="f9e8f9deddf8c0314059bc7c9e56b57d31a25b574d9c00bb706c8b748c36b6db" exitCode=0 Jul 15 20:27:54.294394 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:54.293950 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" event={"ID":"45de28a1-7377-4733-a48c-31624d452f6a","Type":"ContainerDied","Data":"f9e8f9deddf8c0314059bc7c9e56b57d31a25b574d9c00bb706c8b748c36b6db"} Jul 15 20:27:54.294394 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:54.294011 2557 scope.go:117] "RemoveContainer" containerID="5883145c6bfc36999068c7d4b91c9bef602cbb57245245865c88a7eb24fce997" Jul 15 20:27:55.334443 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:55.334408 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:55.432708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:55.432681 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:55.518079 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:55.518048 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca\") pod \"45de28a1-7377-4733-a48c-31624d452f6a\" (UID: \"45de28a1-7377-4733-a48c-31624d452f6a\") " Jul 15 20:27:55.518403 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:55.518382 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45de28a1-7377-4733-a48c-31624d452f6a" (UID: "45de28a1-7377-4733-a48c-31624d452f6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:27:55.619064 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:55.618967 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45de28a1-7377-4733-a48c-31624d452f6a-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:27:56.306863 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:56.306830 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" Jul 15 20:27:56.306863 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:56.306833 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx" event={"ID":"45de28a1-7377-4733-a48c-31624d452f6a","Type":"ContainerDied","Data":"d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045"} Jul 15 20:27:56.307062 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:56.306911 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9294d1251faf5179e90f5c24a70d4fde174c02193dea477329418273501c045" Jul 15 20:27:56.351652 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:56.351618 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:56.360267 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:56.360240 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-6xwpx"] Jul 15 20:27:58.122562 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:27:58.122524 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45de28a1-7377-4733-a48c-31624d452f6a" path="/var/lib/kubelet/pods/45de28a1-7377-4733-a48c-31624d452f6a/volumes" Jul 15 20:28:26.615986 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.615946 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv"] Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616311 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616323 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616340 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616346 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616398 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.616424 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.616406 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:26.619421 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.619404 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:26.622490 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.622470 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:28:26.622606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.622496 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:28:26.629036 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.629010 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv"] Jul 15 20:28:26.697924 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.697892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv\" (UID: \"22a145f9-3553-4d9c-aea1-4f7709988566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:26.799309 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.799268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv\" (UID: \"22a145f9-3553-4d9c-aea1-4f7709988566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:26.799855 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.799835 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv\" (UID: \"22a145f9-3553-4d9c-aea1-4f7709988566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:26.931334 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:26.931236 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:27.062877 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:27.062849 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv"] Jul 15 20:28:27.064372 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:28:27.064345 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a145f9_3553_4d9c_aea1_4f7709988566.slice/crio-0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263 WatchSource:0}: Error finding container 0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263: Status 404 returned error can't find the container with id 0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263 Jul 15 20:28:27.476326 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:27.476283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" event={"ID":"22a145f9-3553-4d9c-aea1-4f7709988566","Type":"ContainerStarted","Data":"36b0fedbf884176fa0d1d301018cff4d98f6c8d6876217386f71ffb2666d29ad"} Jul 15 20:28:27.476502 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:27.476353 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" event={"ID":"22a145f9-3553-4d9c-aea1-4f7709988566","Type":"ContainerStarted","Data":"0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263"} Jul 15 20:28:27.496711 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:27.496663 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" podStartSLOduration=1.496647201 podStartE2EDuration="1.496647201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:28:27.495539388 +0000 UTC m=+1800.367532826" watchObservedRunningTime="2026-07-15 20:28:27.496647201 +0000 UTC m=+1800.368640610" Jul 15 20:28:28.160798 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.160772 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:28:28.162341 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.162319 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:28:28.164520 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.164504 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:28:28.166006 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.165979 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:28:28.483594 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.483508 2557 generic.go:356] "Generic (PLEG): container finished" podID="22a145f9-3553-4d9c-aea1-4f7709988566" containerID="36b0fedbf884176fa0d1d301018cff4d98f6c8d6876217386f71ffb2666d29ad" exitCode=0 Jul 15 20:28:28.483594 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:28.483563 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" event={"ID":"22a145f9-3553-4d9c-aea1-4f7709988566","Type":"ContainerDied","Data":"36b0fedbf884176fa0d1d301018cff4d98f6c8d6876217386f71ffb2666d29ad"} Jul 15 20:28:29.618450 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:29.618419 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:29.725507 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:29.725466 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca\") pod \"22a145f9-3553-4d9c-aea1-4f7709988566\" (UID: \"22a145f9-3553-4d9c-aea1-4f7709988566\") " Jul 15 20:28:29.725874 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:29.725852 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22a145f9-3553-4d9c-aea1-4f7709988566" (UID: "22a145f9-3553-4d9c-aea1-4f7709988566"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:28:29.827015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:29.826970 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a145f9-3553-4d9c-aea1-4f7709988566-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:28:30.495253 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:30.495164 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" Jul 15 20:28:30.495253 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:30.495197 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv" event={"ID":"22a145f9-3553-4d9c-aea1-4f7709988566","Type":"ContainerDied","Data":"0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263"} Jul 15 20:28:30.495253 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:30.495246 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c6e2a83f1569a8988b314e09c7045a4af58aea237b74e1f0040568e747fa263" Jul 15 20:28:39.738312 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:39.738266 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:28:47.753203 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:47.753113 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:28:49.299572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.298554 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:28:49.299572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299378 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:49.299572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299399 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:49.299572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299429 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a145f9-3553-4d9c-aea1-4f7709988566" containerName="revoke-keys" Jul 15 20:28:49.299572 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299436 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a145f9-3553-4d9c-aea1-4f7709988566" containerName="revoke-keys" Jul 15 20:28:49.300252 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299588 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="45de28a1-7377-4733-a48c-31624d452f6a" containerName="revoke-keys" Jul 15 20:28:49.300252 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.299601 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a145f9-3553-4d9c-aea1-4f7709988566" containerName="revoke-keys" Jul 15 20:28:49.303670 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.303643 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:28:49.306657 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.306637 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:28:49.306754 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.306640 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:28:49.312894 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.312859 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:28:49.404132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.404089 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h\" (UID: \"6b12530b-9fe1-463d-aec1-10404f1925ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:28:49.505020 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.504969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h\" (UID: \"6b12530b-9fe1-463d-aec1-10404f1925ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:28:49.505676 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.505649 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h\" (UID: \"6b12530b-9fe1-463d-aec1-10404f1925ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:28:49.615712 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.615616 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:28:49.783198 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:49.783163 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:28:49.784361 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:28:49.784336 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b12530b_9fe1_463d_aec1_10404f1925ae.slice/crio-71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820 WatchSource:0}: Error finding container 71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820: Status 404 returned error can't find the container with id 71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820 Jul 15 20:28:50.603365 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:50.603330 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerID="e1ef2a76ab37e53ad1987abc4f7520a60de1f4554d85391fcfaef68c8c77e02b" exitCode=7 Jul 15 20:28:50.603748 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:50.603418 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerDied","Data":"e1ef2a76ab37e53ad1987abc4f7520a60de1f4554d85391fcfaef68c8c77e02b"} Jul 15 20:28:50.603748 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:50.603477 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerStarted","Data":"71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820"} Jul 15 20:28:50.603748 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:50.603559 2557 scope.go:117] "RemoveContainer" containerID="e1ef2a76ab37e53ad1987abc4f7520a60de1f4554d85391fcfaef68c8c77e02b" Jul 15 20:28:51.610651 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:51.610618 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerID="58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e" exitCode=7 Jul 15 20:28:51.611064 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:51.610680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerDied","Data":"58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e"} Jul 15 20:28:51.611064 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:51.610722 2557 scope.go:117] "RemoveContainer" containerID="e1ef2a76ab37e53ad1987abc4f7520a60de1f4554d85391fcfaef68c8c77e02b" Jul 15 20:28:51.611064 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:51.610892 2557 scope.go:117] "RemoveContainer" containerID="58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e" Jul 15 20:28:51.611230 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:28:51.611141 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h_odh-ai-gateway-infra(6b12530b-9fe1-463d-aec1-10404f1925ae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" Jul 15 20:28:52.618221 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:28:52.618189 2557 scope.go:117] "RemoveContainer" containerID="58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e" Jul 15 20:28:52.618600 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:28:52.618411 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h_odh-ai-gateway-infra(6b12530b-9fe1-463d-aec1-10404f1925ae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" Jul 15 20:29:05.116283 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:05.116252 2557 scope.go:117] "RemoveContainer" containerID="58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e" Jul 15 20:29:05.692131 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:05.692034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerStarted","Data":"7dbb5bb58bf703ec51dd1bdf0b6884381ff11360d7544c37ad7cb016a2418635"} Jul 15 20:29:05.721566 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:05.721509 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" podStartSLOduration=16.721494661 podStartE2EDuration="16.721494661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:28:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:29:05.720086243 +0000 UTC m=+1838.592079671" watchObservedRunningTime="2026-07-15 20:29:05.721494661 +0000 UTC m=+1838.593488091" Jul 15 20:29:06.699476 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:06.699440 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerID="7dbb5bb58bf703ec51dd1bdf0b6884381ff11360d7544c37ad7cb016a2418635" exitCode=0 Jul 15 20:29:06.699476 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:06.699479 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerDied","Data":"7dbb5bb58bf703ec51dd1bdf0b6884381ff11360d7544c37ad7cb016a2418635"} Jul 15 20:29:06.699908 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:06.699527 2557 scope.go:117] "RemoveContainer" containerID="58d70142fedfe5460282605a4b8c80fb80dad2a22180c613e3d75e1dec2dd85e" Jul 15 20:29:06.730465 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:06.730432 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:29:07.838113 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:07.838086 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:29:07.974209 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:07.974111 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca\") pod \"6b12530b-9fe1-463d-aec1-10404f1925ae\" (UID: \"6b12530b-9fe1-463d-aec1-10404f1925ae\") " Jul 15 20:29:07.974472 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:07.974445 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b12530b-9fe1-463d-aec1-10404f1925ae" (UID: "6b12530b-9fe1-463d-aec1-10404f1925ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:29:08.075379 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.075341 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b12530b-9fe1-463d-aec1-10404f1925ae-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:29:08.713507 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.713424 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" Jul 15 20:29:08.713507 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.713462 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h" event={"ID":"6b12530b-9fe1-463d-aec1-10404f1925ae","Type":"ContainerDied","Data":"71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820"} Jul 15 20:29:08.713691 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.713561 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71658dacd4c8b19182cc06141b4653cb56263a67707f2834bd04fc4fc60e8820" Jul 15 20:29:08.766659 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.766626 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:29:08.771634 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:08.771610 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-87t2h"] Jul 15 20:29:10.123269 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:10.123237 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" path="/var/lib/kubelet/pods/6b12530b-9fe1-463d-aec1-10404f1925ae/volumes" Jul 15 20:29:39.349626 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.349580 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98"] Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.349968 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.349979 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.349993 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.349999 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.350009 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.350015 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.350090 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.350097 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.350115 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.350106 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b12530b-9fe1-463d-aec1-10404f1925ae" containerName="revoke-keys" Jul 15 20:29:39.353186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.353168 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:39.356510 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.356487 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:29:39.356642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.356487 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:29:39.363444 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.363416 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98"] Jul 15 20:29:39.450551 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.450513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98\" (UID: \"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:39.551452 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.551410 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98\" (UID: \"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:39.552015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.551996 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98\" (UID: \"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:39.664585 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.664504 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:39.802649 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.802626 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98"] Jul 15 20:29:39.804588 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:29:39.804564 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f10fc1_b2fe_4da1_9299_ecb6acbb60c7.slice/crio-9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c WatchSource:0}: Error finding container 9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c: Status 404 returned error can't find the container with id 9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c Jul 15 20:29:39.889054 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:39.889020 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" event={"ID":"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7","Type":"ContainerStarted","Data":"9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c"} Jul 15 20:29:40.895910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:40.895876 2557 generic.go:356] "Generic (PLEG): container finished" podID="09f10fc1-b2fe-4da1-9299-ecb6acbb60c7" containerID="84fd0f4b50a44061db753529055647c20d6600cf978e0b7d64d15e7df09e8429" exitCode=0 Jul 15 20:29:40.896313 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:40.895964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" event={"ID":"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7","Type":"ContainerDied","Data":"84fd0f4b50a44061db753529055647c20d6600cf978e0b7d64d15e7df09e8429"} Jul 15 20:29:42.039971 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.039941 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:42.074496 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.074448 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca\") pod \"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7\" (UID: \"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7\") " Jul 15 20:29:42.074909 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.074887 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09f10fc1-b2fe-4da1-9299-ecb6acbb60c7" (UID: "09f10fc1-b2fe-4da1-9299-ecb6acbb60c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:29:42.176397 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.176309 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09f10fc1-b2fe-4da1-9299-ecb6acbb60c7-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:29:42.909309 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.909277 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" Jul 15 20:29:42.909492 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.909308 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98" event={"ID":"09f10fc1-b2fe-4da1-9299-ecb6acbb60c7","Type":"ContainerDied","Data":"9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c"} Jul 15 20:29:42.909492 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:42.909360 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9721a2054636c4a66e8b8733b0220ed467c8aceb65ae74911d66709f16a2917c" Jul 15 20:29:51.643874 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:29:51.643779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:30:00.140458 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.140422 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt"] Jul 15 20:30:00.140857 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.140776 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f10fc1-b2fe-4da1-9299-ecb6acbb60c7" containerName="revoke-keys" Jul 15 20:30:00.140857 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.140787 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f10fc1-b2fe-4da1-9299-ecb6acbb60c7" containerName="revoke-keys" Jul 15 20:30:00.140857 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.140849 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f10fc1-b2fe-4da1-9299-ecb6acbb60c7" containerName="revoke-keys" Jul 15 20:30:00.144169 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.144150 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" Jul 15 20:30:00.147051 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.147033 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:30:00.152659 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.152634 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt"] Jul 15 20:30:00.155696 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.155670 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" Jul 15 20:30:00.294690 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:00.293426 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt"] Jul 15 20:30:00.295243 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:30:00.295215 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79a42cd_2937_486f_b964_d43ef587005b.slice/crio-a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a WatchSource:0}: Error finding container a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a: Status 404 returned error can't find the container with id a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a Jul 15 20:30:01.008542 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:01.008510 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" event={"ID":"f79a42cd-2937-486f-b964-d43ef587005b","Type":"ContainerStarted","Data":"23a4d7f16cdca561bca4ada547f3907d8d81cf3d4de91e3d41608afd97d3be73"} Jul 15 20:30:01.008722 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:01.008564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" event={"ID":"f79a42cd-2937-486f-b964-d43ef587005b","Type":"ContainerStarted","Data":"a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a"} Jul 15 20:30:01.025807 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:01.025752 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" podStartSLOduration=1.025737155 podStartE2EDuration="1.025737155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:30:01.02340192 +0000 UTC m=+1893.895395355" watchObservedRunningTime="2026-07-15 20:30:01.025737155 +0000 UTC m=+1893.897730586" Jul 15 20:30:02.015266 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:02.015229 2557 generic.go:356] "Generic (PLEG): container finished" podID="f79a42cd-2937-486f-b964-d43ef587005b" containerID="23a4d7f16cdca561bca4ada547f3907d8d81cf3d4de91e3d41608afd97d3be73" exitCode=0 Jul 15 20:30:02.015677 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:02.015315 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" event={"ID":"f79a42cd-2937-486f-b964-d43ef587005b","Type":"ContainerDied","Data":"23a4d7f16cdca561bca4ada547f3907d8d81cf3d4de91e3d41608afd97d3be73"} Jul 15 20:30:03.161150 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:03.161117 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" Jul 15 20:30:04.028771 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:04.028733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" event={"ID":"f79a42cd-2937-486f-b964-d43ef587005b","Type":"ContainerDied","Data":"a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a"} Jul 15 20:30:04.028942 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:04.028747 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-mrtzt" Jul 15 20:30:04.028942 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:04.028832 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7e556569a62d74d15d24ceffe10d2e8e9da1242735d517cf677fc6bc50c981a" Jul 15 20:30:11.323002 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.322961 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:11.323395 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.323334 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79a42cd-2937-486f-b964-d43ef587005b" containerName="cleanup" Jul 15 20:30:11.323395 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.323345 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79a42cd-2937-486f-b964-d43ef587005b" containerName="cleanup" Jul 15 20:30:11.323467 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.323421 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79a42cd-2937-486f-b964-d43ef587005b" containerName="cleanup" Jul 15 20:30:11.326459 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.326439 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:11.329431 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.329409 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:30:11.329431 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.329422 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:30:11.335960 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.335935 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:11.432597 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.432553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv\" (UID: \"0e79350f-3850-4d34-b0b5-970930ae9013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:11.534148 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.534101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv\" (UID: \"0e79350f-3850-4d34-b0b5-970930ae9013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:11.534741 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.534719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv\" (UID: \"0e79350f-3850-4d34-b0b5-970930ae9013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:11.637638 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.637536 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:11.775857 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:11.775824 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:11.778012 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:30:11.777985 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e79350f_3850_4d34_b0b5_970930ae9013.slice/crio-47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762 WatchSource:0}: Error finding container 47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762: Status 404 returned error can't find the container with id 47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762 Jul 15 20:30:12.075136 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:12.075100 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" event={"ID":"0e79350f-3850-4d34-b0b5-970930ae9013","Type":"ContainerStarted","Data":"47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762"} Jul 15 20:30:13.082399 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:13.082367 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e79350f-3850-4d34-b0b5-970930ae9013" containerID="4bdd88998975f45bebf6b69540ebcb0a4fdfc5f0011509cebfbc41c447d53300" exitCode=7 Jul 15 20:30:13.082804 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:13.082460 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" event={"ID":"0e79350f-3850-4d34-b0b5-970930ae9013","Type":"ContainerDied","Data":"4bdd88998975f45bebf6b69540ebcb0a4fdfc5f0011509cebfbc41c447d53300"} Jul 15 20:30:13.082804 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:13.082657 2557 scope.go:117] "RemoveContainer" containerID="4bdd88998975f45bebf6b69540ebcb0a4fdfc5f0011509cebfbc41c447d53300" Jul 15 20:30:14.089913 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:14.089876 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e79350f-3850-4d34-b0b5-970930ae9013" containerID="af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca" exitCode=7 Jul 15 20:30:14.090361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:14.089962 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" event={"ID":"0e79350f-3850-4d34-b0b5-970930ae9013","Type":"ContainerDied","Data":"af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca"} Jul 15 20:30:14.090361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:14.090020 2557 scope.go:117] "RemoveContainer" containerID="4bdd88998975f45bebf6b69540ebcb0a4fdfc5f0011509cebfbc41c447d53300" Jul 15 20:30:14.090361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:14.090284 2557 scope.go:117] "RemoveContainer" containerID="af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca" Jul 15 20:30:14.090554 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:30:14.090530 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv_odh-ai-gateway-infra(0e79350f-3850-4d34-b0b5-970930ae9013)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" Jul 15 20:30:14.747132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:14.747082 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:30:15.097729 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:15.097701 2557 scope.go:117] "RemoveContainer" containerID="af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca" Jul 15 20:30:15.098215 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:30:15.097943 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv_odh-ai-gateway-infra(0e79350f-3850-4d34-b0b5-970930ae9013)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" Jul 15 20:30:30.116408 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:30.116373 2557 scope.go:117] "RemoveContainer" containerID="af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca" Jul 15 20:30:31.185234 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:31.185130 2557 generic.go:356] "Generic (PLEG): container finished" podID="0e79350f-3850-4d34-b0b5-970930ae9013" containerID="b377898a0dc0e70ae96880a7465430d34953b2853f10c9881af66cdeaa00ba5a" exitCode=0 Jul 15 20:30:31.185234 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:31.185208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" event={"ID":"0e79350f-3850-4d34-b0b5-970930ae9013","Type":"ContainerDied","Data":"b377898a0dc0e70ae96880a7465430d34953b2853f10c9881af66cdeaa00ba5a"} Jul 15 20:30:31.185719 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:31.185296 2557 scope.go:117] "RemoveContainer" containerID="af4c469f119eba78b737fcc6dac1091e2639431d5aa0e8e5482c9bb2bd0419ca" Jul 15 20:30:32.218736 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:32.218703 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:32.342239 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:32.342211 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:32.414315 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:32.414267 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca\") pod \"0e79350f-3850-4d34-b0b5-970930ae9013\" (UID: \"0e79350f-3850-4d34-b0b5-970930ae9013\") " Jul 15 20:30:32.414704 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:32.414676 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e79350f-3850-4d34-b0b5-970930ae9013" (UID: "0e79350f-3850-4d34-b0b5-970930ae9013"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:30:32.515855 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:32.515762 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e79350f-3850-4d34-b0b5-970930ae9013-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:30:33.200221 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:33.200186 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" Jul 15 20:30:33.200409 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:33.200187 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv" event={"ID":"0e79350f-3850-4d34-b0b5-970930ae9013","Type":"ContainerDied","Data":"47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762"} Jul 15 20:30:33.200409 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:33.200322 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47519caaf3c12c0cfcd424c1b86f93cc5d77081f0145d22007db5a4c0d55d762" Jul 15 20:30:33.232585 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:33.232547 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:33.236928 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:33.236900 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-qlrrv"] Jul 15 20:30:34.122665 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:30:34.122627 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" path="/var/lib/kubelet/pods/0e79350f-3850-4d34-b0b5-970930ae9013/volumes" Jul 15 20:31:11.366650 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.366613 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b"] Jul 15 20:31:11.367066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.366987 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.366999 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.367008 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.367013 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367232 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.367122 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367232 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.367132 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.367232 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.367140 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:31:11.370257 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.370238 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:11.373056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.373036 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:31:11.373134 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.373059 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:31:11.380086 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.380045 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b"] Jul 15 20:31:11.469569 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.469526 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b\" (UID: \"79c80666-5d3e-4f1e-ac28-c749cf7de6e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:11.570881 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.570823 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b\" (UID: \"79c80666-5d3e-4f1e-ac28-c749cf7de6e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:11.571514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.571494 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b\" (UID: \"79c80666-5d3e-4f1e-ac28-c749cf7de6e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:11.681644 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.681557 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:11.818435 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:11.818404 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b"] Jul 15 20:31:11.819265 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:31:11.819230 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c80666_5d3e_4f1e_ac28_c749cf7de6e6.slice/crio-6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf WatchSource:0}: Error finding container 6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf: Status 404 returned error can't find the container with id 6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf Jul 15 20:31:12.422322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:12.422284 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" event={"ID":"79c80666-5d3e-4f1e-ac28-c749cf7de6e6","Type":"ContainerStarted","Data":"ef70f67c4bc8ce82b22e7c9003dd6b707725d23252ac82ba617e035f30956739"} Jul 15 20:31:12.422694 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:12.422340 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" event={"ID":"79c80666-5d3e-4f1e-ac28-c749cf7de6e6","Type":"ContainerStarted","Data":"6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf"} Jul 15 20:31:12.440347 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:12.440293 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" podStartSLOduration=1.440275647 podStartE2EDuration="1.440275647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:31:12.438129073 +0000 UTC m=+1965.310122507" watchObservedRunningTime="2026-07-15 20:31:12.440275647 +0000 UTC m=+1965.312269121" Jul 15 20:31:13.429050 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:13.429015 2557 generic.go:356] "Generic (PLEG): container finished" podID="79c80666-5d3e-4f1e-ac28-c749cf7de6e6" containerID="ef70f67c4bc8ce82b22e7c9003dd6b707725d23252ac82ba617e035f30956739" exitCode=0 Jul 15 20:31:13.429481 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:13.429105 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" event={"ID":"79c80666-5d3e-4f1e-ac28-c749cf7de6e6","Type":"ContainerDied","Data":"ef70f67c4bc8ce82b22e7c9003dd6b707725d23252ac82ba617e035f30956739"} Jul 15 20:31:14.572166 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:14.572137 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:14.599056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:14.599027 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca\") pod \"79c80666-5d3e-4f1e-ac28-c749cf7de6e6\" (UID: \"79c80666-5d3e-4f1e-ac28-c749cf7de6e6\") " Jul 15 20:31:14.599409 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:14.599383 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79c80666-5d3e-4f1e-ac28-c749cf7de6e6" (UID: "79c80666-5d3e-4f1e-ac28-c749cf7de6e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:31:14.700648 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:14.700539 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c80666-5d3e-4f1e-ac28-c749cf7de6e6-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:31:15.441140 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:15.441111 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" Jul 15 20:31:15.441335 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:15.441114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b" event={"ID":"79c80666-5d3e-4f1e-ac28-c749cf7de6e6","Type":"ContainerDied","Data":"6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf"} Jul 15 20:31:15.441335 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:15.441254 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6378c09683b0cc733b6a98095bed8affa45beecb0f5508286d513eb0f261fbbf" Jul 15 20:31:24.850560 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:31:24.850477 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:32:28.229922 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:32:28.229891 2557 scope.go:117] "RemoveContainer" containerID="24f27202e27f7a6db9ad4662c87f2f829a3a35c6dd4037b447f870b79661cae2" Jul 15 20:33:01.541962 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:01.541873 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:33:10.452588 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:10.452546 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:33:19.742008 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:19.741966 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:33:28.199949 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.199917 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:33:28.204132 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.204105 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:33:28.207317 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.207291 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:33:28.210841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.210822 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:33:28.504292 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504204 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g"] Jul 15 20:33:28.504574 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504562 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:33:28.504617 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504576 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e79350f-3850-4d34-b0b5-970930ae9013" containerName="revoke-keys" Jul 15 20:33:28.504617 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504586 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c80666-5d3e-4f1e-ac28-c749cf7de6e6" containerName="revoke-keys" Jul 15 20:33:28.504617 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504592 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c80666-5d3e-4f1e-ac28-c749cf7de6e6" containerName="revoke-keys" Jul 15 20:33:28.504715 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.504653 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c80666-5d3e-4f1e-ac28-c749cf7de6e6" containerName="revoke-keys" Jul 15 20:33:28.507594 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.507575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:28.510375 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.510349 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:33:28.510517 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.510363 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:33:28.517666 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.517637 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g"] Jul 15 20:33:28.527254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.527193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g\" (UID: \"4b2d917d-d5ce-4790-9946-ecb8b1373fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:28.628632 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.628590 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g\" (UID: \"4b2d917d-d5ce-4790-9946-ecb8b1373fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:28.629347 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.629320 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g\" (UID: \"4b2d917d-d5ce-4790-9946-ecb8b1373fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:28.819393 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.819358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:28.975211 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:28.975181 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g"] Jul 15 20:33:28.977159 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:33:28.977134 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2d917d_d5ce_4790_9946_ecb8b1373fa4.slice/crio-dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2 WatchSource:0}: Error finding container dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2: Status 404 returned error can't find the container with id dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2 Jul 15 20:33:29.199345 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:29.199260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" event={"ID":"4b2d917d-d5ce-4790-9946-ecb8b1373fa4","Type":"ContainerStarted","Data":"dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2"} Jul 15 20:33:30.206637 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:30.206601 2557 generic.go:356] "Generic (PLEG): container finished" podID="4b2d917d-d5ce-4790-9946-ecb8b1373fa4" containerID="67283c163f323af23e57b8a17c6e3a3873c4e46454acdea9f69ef220ca2284a4" exitCode=0 Jul 15 20:33:30.207017 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:30.206687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" event={"ID":"4b2d917d-d5ce-4790-9946-ecb8b1373fa4","Type":"ContainerDied","Data":"67283c163f323af23e57b8a17c6e3a3873c4e46454acdea9f69ef220ca2284a4"} Jul 15 20:33:31.349949 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:31.349922 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:31.458254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:31.458214 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca\") pod \"4b2d917d-d5ce-4790-9946-ecb8b1373fa4\" (UID: \"4b2d917d-d5ce-4790-9946-ecb8b1373fa4\") " Jul 15 20:33:31.458620 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:31.458595 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b2d917d-d5ce-4790-9946-ecb8b1373fa4" (UID: "4b2d917d-d5ce-4790-9946-ecb8b1373fa4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:33:31.559823 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:31.559780 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d917d-d5ce-4790-9946-ecb8b1373fa4-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:33:32.220989 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:32.220956 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" Jul 15 20:33:32.220989 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:32.220976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g" event={"ID":"4b2d917d-d5ce-4790-9946-ecb8b1373fa4","Type":"ContainerDied","Data":"dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2"} Jul 15 20:33:32.221216 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:32.221026 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd6b47e9ed6476418c4b5fb9b09779e4e45aa243a38d50c0569c2b9162e991d2" Jul 15 20:33:42.240469 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:33:42.240416 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:34:19.646451 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:19.646367 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:34:28.259478 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:28.259442 2557 scope.go:117] "RemoveContainer" containerID="f9e8f9deddf8c0314059bc7c9e56b57d31a25b574d9c00bb706c8b748c36b6db" Jul 15 20:34:37.140990 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:37.140951 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:34:51.450230 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:51.450184 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:34:54.661474 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.661435 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr"] Jul 15 20:34:54.661852 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.661783 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2d917d-d5ce-4790-9946-ecb8b1373fa4" containerName="revoke-keys" Jul 15 20:34:54.661852 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.661794 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2d917d-d5ce-4790-9946-ecb8b1373fa4" containerName="revoke-keys" Jul 15 20:34:54.661929 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.661868 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2d917d-d5ce-4790-9946-ecb8b1373fa4" containerName="revoke-keys" Jul 15 20:34:54.665178 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.665161 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:54.668159 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.668129 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:34:54.668297 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.668170 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:34:54.673600 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.673565 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr"] Jul 15 20:34:54.718429 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.718384 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr\" (UID: \"a5934b82-8968-4a02-a78e-0f4d0e84ca0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:54.819674 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.819641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr\" (UID: \"a5934b82-8968-4a02-a78e-0f4d0e84ca0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:54.820414 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.820392 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr\" (UID: \"a5934b82-8968-4a02-a78e-0f4d0e84ca0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:54.977161 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:54.977035 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:55.115248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:55.115224 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr"] Jul 15 20:34:55.116930 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:34:55.116901 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5934b82_8968_4a02_a78e_0f4d0e84ca0c.slice/crio-80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5 WatchSource:0}: Error finding container 80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5: Status 404 returned error can't find the container with id 80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5 Jul 15 20:34:55.701222 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:55.701122 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" event={"ID":"a5934b82-8968-4a02-a78e-0f4d0e84ca0c","Type":"ContainerStarted","Data":"949b65bf365c94ab5ec435a14e08997b22fba2de3fa09f898e9fb29efb36fb12"} Jul 15 20:34:55.701222 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:55.701183 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" event={"ID":"a5934b82-8968-4a02-a78e-0f4d0e84ca0c","Type":"ContainerStarted","Data":"80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5"} Jul 15 20:34:55.719925 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:55.719868 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" podStartSLOduration=1.7198493959999999 podStartE2EDuration="1.719849396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:34:55.717837777 +0000 UTC m=+2188.589831221" watchObservedRunningTime="2026-07-15 20:34:55.719849396 +0000 UTC m=+2188.591842852" Jul 15 20:34:56.708519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:56.708484 2557 generic.go:356] "Generic (PLEG): container finished" podID="a5934b82-8968-4a02-a78e-0f4d0e84ca0c" containerID="949b65bf365c94ab5ec435a14e08997b22fba2de3fa09f898e9fb29efb36fb12" exitCode=0 Jul 15 20:34:56.708911 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:56.708533 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" event={"ID":"a5934b82-8968-4a02-a78e-0f4d0e84ca0c","Type":"ContainerDied","Data":"949b65bf365c94ab5ec435a14e08997b22fba2de3fa09f898e9fb29efb36fb12"} Jul 15 20:34:57.852495 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:57.852468 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:57.949304 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:57.949265 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca\") pod \"a5934b82-8968-4a02-a78e-0f4d0e84ca0c\" (UID: \"a5934b82-8968-4a02-a78e-0f4d0e84ca0c\") " Jul 15 20:34:57.949640 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:57.949615 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5934b82-8968-4a02-a78e-0f4d0e84ca0c" (UID: "a5934b82-8968-4a02-a78e-0f4d0e84ca0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:34:58.050386 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:58.050354 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5934b82-8968-4a02-a78e-0f4d0e84ca0c-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:34:58.721203 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:58.721166 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" event={"ID":"a5934b82-8968-4a02-a78e-0f4d0e84ca0c","Type":"ContainerDied","Data":"80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5"} Jul 15 20:34:58.721203 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:58.721186 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr" Jul 15 20:34:58.721537 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:34:58.721219 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80a8997f7bc724782b01b75dd9bee90c7ca9674fb027bc094c1358e695490fb5" Jul 15 20:35:08.956947 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:35:08.956909 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:35:28.281463 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:35:28.281430 2557 scope.go:117] "RemoveContainer" containerID="7dbb5bb58bf703ec51dd1bdf0b6884381ff11360d7544c37ad7cb016a2418635" Jul 15 20:36:06.058393 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:36:06.058307 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:36:19.155780 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:36:19.155730 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:37:04.644921 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:37:04.644883 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:37:21.549383 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:37:21.549293 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:37:28.311656 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:37:28.311624 2557 scope.go:117] "RemoveContainer" containerID="b377898a0dc0e70ae96880a7465430d34953b2853f10c9881af66cdeaa00ba5a" Jul 15 20:37:34.001803 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:37:34.001767 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:37:44.948519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:37:44.948484 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:38:07.113342 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.113295 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb"] Jul 15 20:38:07.113789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.113677 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5934b82-8968-4a02-a78e-0f4d0e84ca0c" containerName="revoke-keys" Jul 15 20:38:07.113789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.113689 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5934b82-8968-4a02-a78e-0f4d0e84ca0c" containerName="revoke-keys" Jul 15 20:38:07.113789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.113754 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5934b82-8968-4a02-a78e-0f4d0e84ca0c" containerName="revoke-keys" Jul 15 20:38:07.116795 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.116777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:07.119797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.119778 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:38:07.120005 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.119981 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:38:07.125480 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.125453 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb"] Jul 15 20:38:07.252791 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.252749 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb\" (UID: \"b9178014-79e7-4892-92da-d4ae7557591a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:07.353525 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.353477 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb\" (UID: \"b9178014-79e7-4892-92da-d4ae7557591a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:07.354151 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.354129 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb\" (UID: \"b9178014-79e7-4892-92da-d4ae7557591a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:07.429240 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.429152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:07.569417 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.569385 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb"] Jul 15 20:38:07.570946 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:38:07.570920 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9178014_79e7_4892_92da_d4ae7557591a.slice/crio-5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093 WatchSource:0}: Error finding container 5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093: Status 404 returned error can't find the container with id 5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093 Jul 15 20:38:07.810557 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:07.810523 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" event={"ID":"b9178014-79e7-4892-92da-d4ae7557591a","Type":"ContainerStarted","Data":"5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093"} Jul 15 20:38:08.817051 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:08.817015 2557 generic.go:356] "Generic (PLEG): container finished" podID="b9178014-79e7-4892-92da-d4ae7557591a" containerID="8fd6eb49b9f837b8e07b5e97431e1cb679964f0ea71b0eb9b623f02448a5a696" exitCode=0 Jul 15 20:38:08.817459 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:08.817110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" event={"ID":"b9178014-79e7-4892-92da-d4ae7557591a","Type":"ContainerDied","Data":"8fd6eb49b9f837b8e07b5e97431e1cb679964f0ea71b0eb9b623f02448a5a696"} Jul 15 20:38:09.959482 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:09.959455 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:09.974326 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:09.974294 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca\") pod \"b9178014-79e7-4892-92da-d4ae7557591a\" (UID: \"b9178014-79e7-4892-92da-d4ae7557591a\") " Jul 15 20:38:09.974792 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:09.974752 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9178014-79e7-4892-92da-d4ae7557591a" (UID: "b9178014-79e7-4892-92da-d4ae7557591a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:38:10.075809 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:10.075719 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9178014-79e7-4892-92da-d4ae7557591a-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:38:10.441591 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:10.441485 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:38:10.829937 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:10.829910 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" Jul 15 20:38:10.830167 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:10.829947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb" event={"ID":"b9178014-79e7-4892-92da-d4ae7557591a","Type":"ContainerDied","Data":"5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093"} Jul 15 20:38:10.830167 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:10.829996 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b41b072cb5c2a42b9e3cfdc230bf899541fc7916a81925f96f1b01ca2c7b093" Jul 15 20:38:28.054189 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:28.054152 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:38:28.248672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:28.248637 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:38:28.251168 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:28.251140 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:38:28.252111 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:28.252094 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:38:28.254685 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:28.254668 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:38:33.258789 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.258754 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr"] Jul 15 20:38:33.259254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.259130 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9178014-79e7-4892-92da-d4ae7557591a" containerName="revoke-keys" Jul 15 20:38:33.259254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.259145 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9178014-79e7-4892-92da-d4ae7557591a" containerName="revoke-keys" Jul 15 20:38:33.259254 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.259217 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9178014-79e7-4892-92da-d4ae7557591a" containerName="revoke-keys" Jul 15 20:38:33.262599 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.262580 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:33.265602 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.265579 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:38:33.265728 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.265627 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:38:33.271919 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.271891 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr"] Jul 15 20:38:33.381181 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.381136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr\" (UID: \"d43c3b92-15be-41b1-9ea8-8b359d195ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:33.481763 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.481720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr\" (UID: \"d43c3b92-15be-41b1-9ea8-8b359d195ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:33.482397 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.482376 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr\" (UID: \"d43c3b92-15be-41b1-9ea8-8b359d195ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:33.574361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.574327 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:33.741184 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.741138 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr"] Jul 15 20:38:33.744974 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:38:33.744949 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43c3b92_15be_41b1_9ea8_8b359d195ba2.slice/crio-ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8 WatchSource:0}: Error finding container ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8: Status 404 returned error can't find the container with id ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8 Jul 15 20:38:33.966026 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.965987 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" event={"ID":"d43c3b92-15be-41b1-9ea8-8b359d195ba2","Type":"ContainerStarted","Data":"5dfb37550c75ede26d509b7d83f845c832db28dbad00c5216280085a0f078623"} Jul 15 20:38:33.966259 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.966056 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" event={"ID":"d43c3b92-15be-41b1-9ea8-8b359d195ba2","Type":"ContainerStarted","Data":"ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8"} Jul 15 20:38:33.987431 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:33.987378 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" podStartSLOduration=0.987361503 podStartE2EDuration="987.361503ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:38:33.984558911 +0000 UTC m=+2406.856552342" watchObservedRunningTime="2026-07-15 20:38:33.987361503 +0000 UTC m=+2406.859354935" Jul 15 20:38:34.972967 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:34.972931 2557 generic.go:356] "Generic (PLEG): container finished" podID="d43c3b92-15be-41b1-9ea8-8b359d195ba2" containerID="5dfb37550c75ede26d509b7d83f845c832db28dbad00c5216280085a0f078623" exitCode=0 Jul 15 20:38:34.973400 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:34.973015 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" event={"ID":"d43c3b92-15be-41b1-9ea8-8b359d195ba2","Type":"ContainerDied","Data":"5dfb37550c75ede26d509b7d83f845c832db28dbad00c5216280085a0f078623"} Jul 15 20:38:36.114597 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.114569 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:36.208966 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.208910 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca\") pod \"d43c3b92-15be-41b1-9ea8-8b359d195ba2\" (UID: \"d43c3b92-15be-41b1-9ea8-8b359d195ba2\") " Jul 15 20:38:36.209378 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.209349 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d43c3b92-15be-41b1-9ea8-8b359d195ba2" (UID: "d43c3b92-15be-41b1-9ea8-8b359d195ba2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:38:36.309959 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.309926 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43c3b92-15be-41b1-9ea8-8b359d195ba2-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:38:36.990632 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.990602 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" Jul 15 20:38:36.990632 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.990625 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr" event={"ID":"d43c3b92-15be-41b1-9ea8-8b359d195ba2","Type":"ContainerDied","Data":"ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8"} Jul 15 20:38:36.990866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:36.990676 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff3077b73ae98896113bf7edad0169eb1dd1d2b053bc52f9a868120a6b16bca8" Jul 15 20:38:47.058863 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:38:47.058819 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:39:07.475133 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:07.475034 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:39:11.484733 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.484680 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:39:11.485857 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.485831 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43c3b92-15be-41b1-9ea8-8b359d195ba2" containerName="revoke-keys" Jul 15 20:39:11.485980 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.485969 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43c3b92-15be-41b1-9ea8-8b359d195ba2" containerName="revoke-keys" Jul 15 20:39:11.486173 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.486160 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43c3b92-15be-41b1-9ea8-8b359d195ba2" containerName="revoke-keys" Jul 15 20:39:11.489637 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.489613 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.492739 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.492716 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-44c9f6-serving-cert\"" Jul 15 20:39:11.493208 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.493191 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:39:11.493281 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.493191 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 20:39:11.494062 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.494047 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z7fnv\"" Jul 15 20:39:11.501319 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.501291 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:39:11.535608 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.535570 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.535608 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.535608 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctch2\" (UniqueName: \"kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.535825 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.535639 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.636259 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.636220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.636259 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.636259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctch2\" (UniqueName: \"kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.636468 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.636301 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.636506 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:39:11.636481 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-serving-cert: secret "maas-api-e2e-api-b-44c9f6-serving-cert" not found Jul 15 20:39:11.636567 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:39:11.636556 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls podName:93454668-741d-47d2-ad61-c662420e55a1 nodeName:}" failed. No retries permitted until 2026-07-15 20:39:12.136534393 +0000 UTC m=+2445.008527803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls") pod "maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" (UID: "93454668-741d-47d2-ad61-c662420e55a1") : secret "maas-api-e2e-api-b-44c9f6-serving-cert" not found Jul 15 20:39:11.636943 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.636925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:11.645473 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:11.645440 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctch2\" (UniqueName: \"kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:12.142733 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:12.142684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:12.145474 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:12.145433 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") pod \"maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:12.405306 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:12.405190 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:12.556884 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:12.556856 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:39:12.557896 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:39:12.557868 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93454668_741d_47d2_ad61_c662420e55a1.slice/crio-c7025ebf4de8a817bdc7d5f745afd2c279afa2dcff798d1f326b43f8a18c3186 WatchSource:0}: Error finding container c7025ebf4de8a817bdc7d5f745afd2c279afa2dcff798d1f326b43f8a18c3186: Status 404 returned error can't find the container with id c7025ebf4de8a817bdc7d5f745afd2c279afa2dcff798d1f326b43f8a18c3186 Jul 15 20:39:12.559279 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:12.559263 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:39:13.197993 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:13.197947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" event={"ID":"93454668-741d-47d2-ad61-c662420e55a1","Type":"ContainerStarted","Data":"c7025ebf4de8a817bdc7d5f745afd2c279afa2dcff798d1f326b43f8a18c3186"} Jul 15 20:39:15.211619 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:15.211521 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" event={"ID":"93454668-741d-47d2-ad61-c662420e55a1","Type":"ContainerStarted","Data":"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec"} Jul 15 20:39:15.212057 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:15.211649 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:15.230425 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:15.230374 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" podStartSLOduration=1.928413666 podStartE2EDuration="4.230356999s" totalImagesPullingTime="2.301943333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:39:15.229125347 +0000 UTC m=+2448.101118779" watchObservedRunningTime="2026-07-15 20:39:15.230356999 +0000 UTC m=+2448.102350431" Jul 15 20:39:20.572388 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:20.572350 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:39:21.223712 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:21.223682 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:39:24.009581 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.009497 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2"] Jul 15 20:39:24.013258 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.013233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:24.016171 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.016148 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:39:24.022401 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.022373 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2"] Jul 15 20:39:24.153998 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.153964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2\" (UID: \"01a0de51-c8bc-4368-aaca-cf1b527f829b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:24.255274 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.255239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2\" (UID: \"01a0de51-c8bc-4368-aaca-cf1b527f829b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:24.255936 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.255915 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2\" (UID: \"01a0de51-c8bc-4368-aaca-cf1b527f829b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:24.325095 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.325035 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:24.468890 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:24.468862 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2"] Jul 15 20:39:24.470375 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:39:24.470346 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a0de51_c8bc_4368_aaca_cf1b527f829b.slice/crio-0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26 WatchSource:0}: Error finding container 0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26: Status 404 returned error can't find the container with id 0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26 Jul 15 20:39:25.272060 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:25.272019 2557 generic.go:356] "Generic (PLEG): container finished" podID="01a0de51-c8bc-4368-aaca-cf1b527f829b" containerID="991d1271c30bc577d8aa5007ef59e4c9a2de69c9694ef55d00efae4dc2083fd3" exitCode=0 Jul 15 20:39:25.272455 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:25.272115 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" event={"ID":"01a0de51-c8bc-4368-aaca-cf1b527f829b","Type":"ContainerDied","Data":"991d1271c30bc577d8aa5007ef59e4c9a2de69c9694ef55d00efae4dc2083fd3"} Jul 15 20:39:25.272455 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:25.272205 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" event={"ID":"01a0de51-c8bc-4368-aaca-cf1b527f829b","Type":"ContainerStarted","Data":"0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26"} Jul 15 20:39:26.407247 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:26.407217 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:26.473942 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:26.473909 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca\") pod \"01a0de51-c8bc-4368-aaca-cf1b527f829b\" (UID: \"01a0de51-c8bc-4368-aaca-cf1b527f829b\") " Jul 15 20:39:26.474334 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:26.474311 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01a0de51-c8bc-4368-aaca-cf1b527f829b" (UID: "01a0de51-c8bc-4368-aaca-cf1b527f829b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:39:26.575672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:26.575635 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a0de51-c8bc-4368-aaca-cf1b527f829b-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:39:27.284613 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:27.284580 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" Jul 15 20:39:27.284786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:27.284611 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2" event={"ID":"01a0de51-c8bc-4368-aaca-cf1b527f829b","Type":"ContainerDied","Data":"0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26"} Jul 15 20:39:27.284786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:27.284653 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d142fdd9b1dbc9940e9d62650f30806139a2c1f240392e993b24ce6c22dee26" Jul 15 20:39:39.647304 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:39.647265 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:39:50.171590 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.171553 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds"] Jul 15 20:39:50.171977 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.171929 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a0de51-c8bc-4368-aaca-cf1b527f829b" containerName="revoke-keys" Jul 15 20:39:50.171977 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.171940 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a0de51-c8bc-4368-aaca-cf1b527f829b" containerName="revoke-keys" Jul 15 20:39:50.172050 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.172003 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a0de51-c8bc-4368-aaca-cf1b527f829b" containerName="revoke-keys" Jul 15 20:39:50.178163 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.178142 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:50.181516 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.181486 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:39:50.184907 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.184878 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds"] Jul 15 20:39:50.196539 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.196499 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds\" (UID: \"d5c878a4-2a47-490e-aa82-a2c2af77935c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:50.297910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.297875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds\" (UID: \"d5c878a4-2a47-490e-aa82-a2c2af77935c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:50.298547 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.298524 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds\" (UID: \"d5c878a4-2a47-490e-aa82-a2c2af77935c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:50.490750 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.490659 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:50.635558 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:50.635529 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds"] Jul 15 20:39:50.637572 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:39:50.637542 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c878a4_2a47_490e_aa82_a2c2af77935c.slice/crio-dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503 WatchSource:0}: Error finding container dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503: Status 404 returned error can't find the container with id dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503 Jul 15 20:39:51.425964 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:51.425919 2557 generic.go:356] "Generic (PLEG): container finished" podID="d5c878a4-2a47-490e-aa82-a2c2af77935c" containerID="3400df93679c08ae41fac004f94a4054de56d93ee5515c813881059ecb3bebb2" exitCode=0 Jul 15 20:39:51.426361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:51.426002 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" event={"ID":"d5c878a4-2a47-490e-aa82-a2c2af77935c","Type":"ContainerDied","Data":"3400df93679c08ae41fac004f94a4054de56d93ee5515c813881059ecb3bebb2"} Jul 15 20:39:51.426361 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:51.426086 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" event={"ID":"d5c878a4-2a47-490e-aa82-a2c2af77935c","Type":"ContainerStarted","Data":"dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503"} Jul 15 20:39:52.566028 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:52.566001 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:52.617427 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:52.617393 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca\") pod \"d5c878a4-2a47-490e-aa82-a2c2af77935c\" (UID: \"d5c878a4-2a47-490e-aa82-a2c2af77935c\") " Jul 15 20:39:52.617744 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:52.617721 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5c878a4-2a47-490e-aa82-a2c2af77935c" (UID: "d5c878a4-2a47-490e-aa82-a2c2af77935c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:39:52.718985 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:52.718894 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c878a4-2a47-490e-aa82-a2c2af77935c-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:39:53.438888 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:53.438854 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" Jul 15 20:39:53.439098 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:53.438893 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds" event={"ID":"d5c878a4-2a47-490e-aa82-a2c2af77935c","Type":"ContainerDied","Data":"dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503"} Jul 15 20:39:53.439098 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:39:53.438942 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd5fc96c1009781e72825d4455ce204868da9144b1522f7872cdee18d59ef503" Jul 15 20:40:00.662402 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.662365 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:40:00.662817 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.662585 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" podUID="93454668-741d-47d2-ad61-c662420e55a1" containerName="maas-api" containerID="cri-o://34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec" gracePeriod=30 Jul 15 20:40:00.914320 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.914254 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:40:00.995672 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.995635 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca\") pod \"93454668-741d-47d2-ad61-c662420e55a1\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " Jul 15 20:40:00.995879 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.995685 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctch2\" (UniqueName: \"kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2\") pod \"93454668-741d-47d2-ad61-c662420e55a1\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " Jul 15 20:40:00.995879 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.995745 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") pod \"93454668-741d-47d2-ad61-c662420e55a1\" (UID: \"93454668-741d-47d2-ad61-c662420e55a1\") " Jul 15 20:40:00.996152 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.996123 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93454668-741d-47d2-ad61-c662420e55a1" (UID: "93454668-741d-47d2-ad61-c662420e55a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:40:00.998066 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.998039 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2" (OuterVolumeSpecName: "kube-api-access-ctch2") pod "93454668-741d-47d2-ad61-c662420e55a1" (UID: "93454668-741d-47d2-ad61-c662420e55a1"). InnerVolumeSpecName "kube-api-access-ctch2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:40:00.998169 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:00.998062 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93454668-741d-47d2-ad61-c662420e55a1" (UID: "93454668-741d-47d2-ad61-c662420e55a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:40:01.096716 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.096656 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93454668-741d-47d2-ad61-c662420e55a1-maas-api-tls\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:40:01.096716 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.096696 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93454668-741d-47d2-ad61-c662420e55a1-openshift-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:40:01.096716 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.096712 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctch2\" (UniqueName: \"kubernetes.io/projected/93454668-741d-47d2-ad61-c662420e55a1-kube-api-access-ctch2\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:40:01.490419 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.490380 2557 generic.go:356] "Generic (PLEG): container finished" podID="93454668-741d-47d2-ad61-c662420e55a1" containerID="34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec" exitCode=0 Jul 15 20:40:01.490628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.490435 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" Jul 15 20:40:01.490628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.490465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" event={"ID":"93454668-741d-47d2-ad61-c662420e55a1","Type":"ContainerDied","Data":"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec"} Jul 15 20:40:01.490628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.490562 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8" event={"ID":"93454668-741d-47d2-ad61-c662420e55a1","Type":"ContainerDied","Data":"c7025ebf4de8a817bdc7d5f745afd2c279afa2dcff798d1f326b43f8a18c3186"} Jul 15 20:40:01.490628 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.490593 2557 scope.go:117] "RemoveContainer" containerID="34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec" Jul 15 20:40:01.500800 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.500777 2557 scope.go:117] "RemoveContainer" containerID="34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec" Jul 15 20:40:01.501119 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:40:01.501056 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec\": container with ID starting with 34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec not found: ID does not exist" containerID="34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec" Jul 15 20:40:01.501230 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.501116 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec"} err="failed to get container status \"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec\": rpc error: code = NotFound desc = could not find container \"34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec\": container with ID starting with 34a7b45aad151dfe82a41b5094b0645ab49a5076dbf1a5a515af0fc2bdb053ec not found: ID does not exist" Jul 15 20:40:01.514291 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.514259 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:40:01.520304 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:01.520278 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c9f6-7bdc5b698b-lbdd8"] Jul 15 20:40:02.121867 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:02.121831 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93454668-741d-47d2-ad61-c662420e55a1" path="/var/lib/kubelet/pods/93454668-741d-47d2-ad61-c662420e55a1/volumes" Jul 15 20:40:03.943778 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:03.943738 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:40:19.838689 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.838601 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.838963 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93454668-741d-47d2-ad61-c662420e55a1" containerName="maas-api" Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.838973 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="93454668-741d-47d2-ad61-c662420e55a1" containerName="maas-api" Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.838993 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c878a4-2a47-490e-aa82-a2c2af77935c" containerName="revoke-keys" Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.838999 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c878a4-2a47-490e-aa82-a2c2af77935c" containerName="revoke-keys" Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.839056 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="93454668-741d-47d2-ad61-c662420e55a1" containerName="maas-api" Jul 15 20:40:19.841289 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.839064 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c878a4-2a47-490e-aa82-a2c2af77935c" containerName="revoke-keys" Jul 15 20:40:19.841969 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.841952 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:19.844845 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.844800 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 20:40:19.844845 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.844840 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-724347-serving-cert\"" Jul 15 20:40:19.845012 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.844809 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:40:19.845012 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.844903 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z7fnv\"" Jul 15 20:40:19.860870 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.860839 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:40:19.979147 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.979103 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:19.979147 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.979151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:19.979379 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:19.979171 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9hkm\" (UniqueName: \"kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.080322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.080277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.080322 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.080319 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.080542 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.080351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9hkm\" (UniqueName: \"kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.080542 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:40:20.080435 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-serving-cert: secret "maas-api-e2e-shared-a-724347-serving-cert" not found Jul 15 20:40:20.080542 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:40:20.080519 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls podName:c83c4e4d-3119-4c18-8d12-df91d097fb22 nodeName:}" failed. No retries permitted until 2026-07-15 20:40:20.580496176 +0000 UTC m=+2513.452489603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls") pod "maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" (UID: "c83c4e4d-3119-4c18-8d12-df91d097fb22") : secret "maas-api-e2e-shared-a-724347-serving-cert" not found Jul 15 20:40:20.080957 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.080937 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.091834 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.091764 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9hkm\" (UniqueName: \"kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.585118 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.585042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.587512 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.587481 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") pod \"maas-api-e2e-shared-a-724347-54bbb76f95-ptm69\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.752638 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.752595 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:20.904261 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:20.904235 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:40:20.905623 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:40:20.905600 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83c4e4d_3119_4c18_8d12_df91d097fb22.slice/crio-0eb7beb7234ed0ca715f2c9cf52e5fc4de747b3fc464046ebe62344100167eaf WatchSource:0}: Error finding container 0eb7beb7234ed0ca715f2c9cf52e5fc4de747b3fc464046ebe62344100167eaf: Status 404 returned error can't find the container with id 0eb7beb7234ed0ca715f2c9cf52e5fc4de747b3fc464046ebe62344100167eaf Jul 15 20:40:21.607955 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:21.607920 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" event={"ID":"c83c4e4d-3119-4c18-8d12-df91d097fb22","Type":"ContainerStarted","Data":"0eb7beb7234ed0ca715f2c9cf52e5fc4de747b3fc464046ebe62344100167eaf"} Jul 15 20:40:22.615508 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:22.615469 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" event={"ID":"c83c4e4d-3119-4c18-8d12-df91d097fb22","Type":"ContainerStarted","Data":"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094"} Jul 15 20:40:22.615928 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:22.615586 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:22.635400 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:22.635343 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" podStartSLOduration=2.970287897 podStartE2EDuration="3.635328681s" totalImagesPullingTime="665.040784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:40:22.633600771 +0000 UTC m=+2515.505594204" watchObservedRunningTime="2026-07-15 20:40:22.635328681 +0000 UTC m=+2515.507322113" Jul 15 20:40:25.953519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:25.953479 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:40:27.927510 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.927474 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:40:27.932189 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.932169 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:27.934980 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.934956 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-5292f5-serving-cert\"" Jul 15 20:40:27.945606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.945578 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:40:27.947014 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.946990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9p7\" (UniqueName: \"kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:27.947140 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.947023 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:27.947199 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:27.947181 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.048119 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.048061 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.048119 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.048118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9p7\" (UniqueName: \"kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.048368 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.048140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.048744 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.048722 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.050960 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.050941 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-5292f5-serving-cert\"" Jul 15 20:40:28.058946 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:40:28.058920 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-serving-cert: secret "maas-api-e2e-shared-b-5292f5-serving-cert" not found Jul 15 20:40:28.059091 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:40:28.059008 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls podName:f8f526d2-860f-4b51-9aed-973ac2f175dc nodeName:}" failed. No retries permitted until 2026-07-15 20:40:28.558986888 +0000 UTC m=+2521.430980313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls") pod "maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" (UID: "f8f526d2-860f-4b51-9aed-973ac2f175dc") : secret "maas-api-e2e-shared-b-5292f5-serving-cert" not found Jul 15 20:40:28.062695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.062669 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9p7\" (UniqueName: \"kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.627936 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.627844 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:40:28.651519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.651475 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.654227 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.654194 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") pod \"maas-api-e2e-shared-b-5292f5-74746976d8-c2r98\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:28.842873 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:28.842832 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:29.012796 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:29.012765 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:40:29.015127 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:40:29.015099 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f526d2_860f_4b51_9aed_973ac2f175dc.slice/crio-54052c496f68d12f91e5b961a67f14c27b14b295517716e330558301205b80b0 WatchSource:0}: Error finding container 54052c496f68d12f91e5b961a67f14c27b14b295517716e330558301205b80b0: Status 404 returned error can't find the container with id 54052c496f68d12f91e5b961a67f14c27b14b295517716e330558301205b80b0 Jul 15 20:40:29.659143 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:29.659020 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" event={"ID":"f8f526d2-860f-4b51-9aed-973ac2f175dc","Type":"ContainerStarted","Data":"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7"} Jul 15 20:40:29.659143 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:29.659106 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" event={"ID":"f8f526d2-860f-4b51-9aed-973ac2f175dc","Type":"ContainerStarted","Data":"54052c496f68d12f91e5b961a67f14c27b14b295517716e330558301205b80b0"} Jul 15 20:40:29.659367 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:29.659231 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:29.692674 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:29.692613 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" podStartSLOduration=2.373110325 podStartE2EDuration="2.692595647s" totalImagesPullingTime="319.485322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:40:29.692311243 +0000 UTC m=+2522.564304676" watchObservedRunningTime="2026-07-15 20:40:29.692595647 +0000 UTC m=+2522.564589080" Jul 15 20:40:35.671186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:35.671150 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:40:42.856056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:40:42.856016 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:41:16.250701 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:41:16.250659 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:41:31.144534 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:41:31.144500 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:41:55.047423 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:41:55.047339 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:42:14.152954 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:42:14.152915 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:42:29.377945 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:42:29.377911 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:42:44.857528 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:42:44.857488 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:42:59.049389 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:42:59.049349 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:43:17.539833 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:17.539786 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:43:28.289344 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:28.289260 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:43:28.291798 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:28.291770 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:43:28.293045 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:28.293019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:43:28.295560 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:28.295541 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:43:31.943165 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:31.943131 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:43:50.043309 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:43:50.043270 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:44:19.845194 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:44:19.845156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:44:35.049257 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:44:35.049213 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:44:54.659790 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:44:54.659701 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:45:00.139035 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.138991 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5"] Jul 15 20:45:00.143020 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.142998 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn"] Jul 15 20:45:00.143186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.143142 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" Jul 15 20:45:00.145866 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.145844 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:45:00.146312 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.146296 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c"] Jul 15 20:45:00.146391 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.146358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" Jul 15 20:45:00.149608 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.149392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" Jul 15 20:45:00.154333 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.154308 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5"] Jul 15 20:45:00.158420 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.158357 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c"] Jul 15 20:45:00.158608 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.158543 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" Jul 15 20:45:00.162708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.162106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn"] Jul 15 20:45:00.171186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.171164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" Jul 15 20:45:00.210782 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.210724 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" Jul 15 20:45:00.331911 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.331873 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5"] Jul 15 20:45:00.334719 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:45:00.334229 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4b2f2e_551c_42b8_8439_f9acffdc4ec0.slice/crio-81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a WatchSource:0}: Error finding container 81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a: Status 404 returned error can't find the container with id 81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a Jul 15 20:45:00.362760 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.362726 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn"] Jul 15 20:45:00.375305 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:45:00.375279 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbf0086_15ba_4e34_b635_f15d09c4284f.slice/crio-174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e WatchSource:0}: Error finding container 174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e: Status 404 returned error can't find the container with id 174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e Jul 15 20:45:00.390105 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:00.390065 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c"] Jul 15 20:45:00.391203 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:45:00.391176 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70cb3f86_18b0_4547_9b95_c65ca312607b.slice/crio-c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441 WatchSource:0}: Error finding container c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441: Status 404 returned error can't find the container with id c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441 Jul 15 20:45:01.238403 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.238362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" event={"ID":"70cb3f86-18b0-4547-9b95-c65ca312607b","Type":"ContainerStarted","Data":"605d9e597a0dbf8fb543673c42edceea1fe8a42e2d177bcb21c3fae4198041fe"} Jul 15 20:45:01.238849 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.238423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" event={"ID":"70cb3f86-18b0-4547-9b95-c65ca312607b","Type":"ContainerStarted","Data":"c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441"} Jul 15 20:45:01.240015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.239971 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" event={"ID":"ecbf0086-15ba-4e34-b635-f15d09c4284f","Type":"ContainerStarted","Data":"cabbb976668e278b56b9df2c79758edac83bb04c9bfd5630184da310f49d835c"} Jul 15 20:45:01.240145 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.240042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" event={"ID":"ecbf0086-15ba-4e34-b635-f15d09c4284f","Type":"ContainerStarted","Data":"174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e"} Jul 15 20:45:01.241462 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.241437 2557 generic.go:356] "Generic (PLEG): container finished" podID="8a4b2f2e-551c-42b8-8439-f9acffdc4ec0" containerID="a548fda7f01139e5f2bf5eab1a0aabde7105017f11ea225bff25dd07b0a59d53" exitCode=0 Jul 15 20:45:01.241563 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.241502 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" event={"ID":"8a4b2f2e-551c-42b8-8439-f9acffdc4ec0","Type":"ContainerDied","Data":"a548fda7f01139e5f2bf5eab1a0aabde7105017f11ea225bff25dd07b0a59d53"} Jul 15 20:45:01.241563 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.241544 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" event={"ID":"8a4b2f2e-551c-42b8-8439-f9acffdc4ec0","Type":"ContainerStarted","Data":"81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a"} Jul 15 20:45:01.258373 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.258324 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" podStartSLOduration=1.258309547 podStartE2EDuration="1.258309547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:45:01.255780878 +0000 UTC m=+2794.127774317" watchObservedRunningTime="2026-07-15 20:45:01.258309547 +0000 UTC m=+2794.130302980" Jul 15 20:45:01.290025 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:01.289968 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" podStartSLOduration=1.2899476650000001 podStartE2EDuration="1.289947665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:45:01.27180249 +0000 UTC m=+2794.143795923" watchObservedRunningTime="2026-07-15 20:45:01.289947665 +0000 UTC m=+2794.161941098" Jul 15 20:45:02.248682 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:02.248647 2557 generic.go:356] "Generic (PLEG): container finished" podID="ecbf0086-15ba-4e34-b635-f15d09c4284f" containerID="cabbb976668e278b56b9df2c79758edac83bb04c9bfd5630184da310f49d835c" exitCode=0 Jul 15 20:45:02.249143 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:02.248728 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" event={"ID":"ecbf0086-15ba-4e34-b635-f15d09c4284f","Type":"ContainerDied","Data":"cabbb976668e278b56b9df2c79758edac83bb04c9bfd5630184da310f49d835c"} Jul 15 20:45:02.250268 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:02.250248 2557 generic.go:356] "Generic (PLEG): container finished" podID="70cb3f86-18b0-4547-9b95-c65ca312607b" containerID="605d9e597a0dbf8fb543673c42edceea1fe8a42e2d177bcb21c3fae4198041fe" exitCode=0 Jul 15 20:45:02.250358 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:02.250310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" event={"ID":"70cb3f86-18b0-4547-9b95-c65ca312607b","Type":"ContainerDied","Data":"605d9e597a0dbf8fb543673c42edceea1fe8a42e2d177bcb21c3fae4198041fe"} Jul 15 20:45:02.393112 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:02.393062 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" Jul 15 20:45:03.258250 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:03.258213 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" Jul 15 20:45:03.258250 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:03.258222 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5" event={"ID":"8a4b2f2e-551c-42b8-8439-f9acffdc4ec0","Type":"ContainerDied","Data":"81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a"} Jul 15 20:45:03.258757 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:03.258298 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81c1c13a92b0ff506e331e94feb73664fb0d1b69532630eca2a91a1e16eeec3a" Jul 15 20:45:03.406231 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:03.406207 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" Jul 15 20:45:03.450466 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:03.450433 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" Jul 15 20:45:04.266249 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.266216 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" Jul 15 20:45:04.266702 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.266253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c" event={"ID":"70cb3f86-18b0-4547-9b95-c65ca312607b","Type":"ContainerDied","Data":"c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441"} Jul 15 20:45:04.266702 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.266300 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c64e52bd37f0cf4a9a37de4836cd852246c0f07a6abda6f1ad8b570d0545f441" Jul 15 20:45:04.267493 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.267472 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" event={"ID":"ecbf0086-15ba-4e34-b635-f15d09c4284f","Type":"ContainerDied","Data":"174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e"} Jul 15 20:45:04.267604 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.267510 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="174f2911e15564b6fb628b2d31ecd53ef7d5fa7d27f6fa509f183cd57ddf374e" Jul 15 20:45:04.267604 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:04.267517 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735805-kqxrn" Jul 15 20:45:14.550821 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:14.550781 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:45:26.148788 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:45:26.148749 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:46:05.943912 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:46:05.943863 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:46:26.348578 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:46:26.348491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:46:42.978156 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:46:42.978111 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:46:57.850365 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:46:57.850317 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:47:09.911301 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911256 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58"] Jul 15 20:47:09.911712 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911700 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70cb3f86-18b0-4547-9b95-c65ca312607b" containerName="cleanup" Jul 15 20:47:09.911712 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911713 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cb3f86-18b0-4547-9b95-c65ca312607b" containerName="cleanup" Jul 15 20:47:09.911786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911721 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbf0086-15ba-4e34-b635-f15d09c4284f" containerName="cleanup" Jul 15 20:47:09.911786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911728 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbf0086-15ba-4e34-b635-f15d09c4284f" containerName="cleanup" Jul 15 20:47:09.911786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911740 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a4b2f2e-551c-42b8-8439-f9acffdc4ec0" containerName="cleanup" Jul 15 20:47:09.911786 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911747 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a4b2f2e-551c-42b8-8439-f9acffdc4ec0" containerName="cleanup" Jul 15 20:47:09.911910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911821 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="70cb3f86-18b0-4547-9b95-c65ca312607b" containerName="cleanup" Jul 15 20:47:09.911910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911828 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a4b2f2e-551c-42b8-8439-f9acffdc4ec0" containerName="cleanup" Jul 15 20:47:09.911910 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.911836 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbf0086-15ba-4e34-b635-f15d09c4284f" containerName="cleanup" Jul 15 20:47:09.915023 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.915000 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:09.917962 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.917931 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:47:09.923847 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:09.923817 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58"] Jul 15 20:47:10.042650 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:10.042606 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-724347-vsh58\" (UID: \"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:10.143432 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:10.143396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-724347-vsh58\" (UID: \"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:10.143998 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:10.143979 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-724347-vsh58\" (UID: \"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:10.228029 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:10.227943 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:10.367899 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:10.367872 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58"] Jul 15 20:47:10.369256 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:47:10.369227 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8e49d0_b255_4b1c_8537_0e0d73ea59c5.slice/crio-4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4 WatchSource:0}: Error finding container 4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4: Status 404 returned error can't find the container with id 4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4 Jul 15 20:47:11.023696 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:11.023601 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" event={"ID":"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5","Type":"ContainerStarted","Data":"7e7ad9a798588b039391c9524ad4812ff18ccd937be8393e03f2824b466a1879"} Jul 15 20:47:11.023696 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:11.023666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" event={"ID":"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5","Type":"ContainerStarted","Data":"4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4"} Jul 15 20:47:11.044683 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:11.044630 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" podStartSLOduration=2.0446104959999998 podStartE2EDuration="2.044610496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:47:11.042242061 +0000 UTC m=+2923.914235492" watchObservedRunningTime="2026-07-15 20:47:11.044610496 +0000 UTC m=+2923.916603927" Jul 15 20:47:12.031663 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:12.031628 2557 generic.go:356] "Generic (PLEG): container finished" podID="8c8e49d0-b255-4b1c-8537-0e0d73ea59c5" containerID="7e7ad9a798588b039391c9524ad4812ff18ccd937be8393e03f2824b466a1879" exitCode=0 Jul 15 20:47:12.032047 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:12.031727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" event={"ID":"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5","Type":"ContainerDied","Data":"7e7ad9a798588b039391c9524ad4812ff18ccd937be8393e03f2824b466a1879"} Jul 15 20:47:13.179905 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:13.179880 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:13.273259 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:13.273206 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca\") pod \"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5\" (UID: \"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5\") " Jul 15 20:47:13.273609 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:13.273581 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c8e49d0-b255-4b1c-8537-0e0d73ea59c5" (UID: "8c8e49d0-b255-4b1c-8537-0e0d73ea59c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:47:13.374317 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:13.374218 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e49d0-b255-4b1c-8537-0e0d73ea59c5-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:14.044824 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:14.044791 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" Jul 15 20:47:14.045096 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:14.044828 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-724347-vsh58" event={"ID":"8c8e49d0-b255-4b1c-8537-0e0d73ea59c5","Type":"ContainerDied","Data":"4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4"} Jul 15 20:47:14.045096 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:14.044875 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f6e2f3ba8334e02e80595649a8ff2119c9dc667936c7e9e04faab625d17aeb4" Jul 15 20:47:20.313695 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.313649 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:47:20.314186 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.313989 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" podUID="c83c4e4d-3119-4c18-8d12-df91d097fb22" containerName="maas-api" containerID="cri-o://2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094" gracePeriod=30 Jul 15 20:47:20.333877 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.333846 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5"] Jul 15 20:47:20.341998 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.341964 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-724347-29735805-82qx5"] Jul 15 20:47:20.570426 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.570352 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:47:20.641414 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.641374 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9hkm\" (UniqueName: \"kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm\") pod \"c83c4e4d-3119-4c18-8d12-df91d097fb22\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " Jul 15 20:47:20.641627 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.641452 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca\") pod \"c83c4e4d-3119-4c18-8d12-df91d097fb22\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " Jul 15 20:47:20.641627 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.641478 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") pod \"c83c4e4d-3119-4c18-8d12-df91d097fb22\" (UID: \"c83c4e4d-3119-4c18-8d12-df91d097fb22\") " Jul 15 20:47:20.641877 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.641851 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c83c4e4d-3119-4c18-8d12-df91d097fb22" (UID: "c83c4e4d-3119-4c18-8d12-df91d097fb22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:47:20.643571 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.643544 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c83c4e4d-3119-4c18-8d12-df91d097fb22" (UID: "c83c4e4d-3119-4c18-8d12-df91d097fb22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:47:20.643639 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.643616 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm" (OuterVolumeSpecName: "kube-api-access-r9hkm") pod "c83c4e4d-3119-4c18-8d12-df91d097fb22" (UID: "c83c4e4d-3119-4c18-8d12-df91d097fb22"). InnerVolumeSpecName "kube-api-access-r9hkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:47:20.742121 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.742049 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83c4e4d-3119-4c18-8d12-df91d097fb22-openshift-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:20.742121 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.742115 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c83c4e4d-3119-4c18-8d12-df91d097fb22-maas-api-tls\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:20.742121 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:20.742125 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9hkm\" (UniqueName: \"kubernetes.io/projected/c83c4e4d-3119-4c18-8d12-df91d097fb22-kube-api-access-r9hkm\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:21.088752 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.088719 2557 generic.go:356] "Generic (PLEG): container finished" podID="c83c4e4d-3119-4c18-8d12-df91d097fb22" containerID="2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094" exitCode=0 Jul 15 20:47:21.088940 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.088765 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" Jul 15 20:47:21.088940 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.088808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" event={"ID":"c83c4e4d-3119-4c18-8d12-df91d097fb22","Type":"ContainerDied","Data":"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094"} Jul 15 20:47:21.088940 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.088861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69" event={"ID":"c83c4e4d-3119-4c18-8d12-df91d097fb22","Type":"ContainerDied","Data":"0eb7beb7234ed0ca715f2c9cf52e5fc4de747b3fc464046ebe62344100167eaf"} Jul 15 20:47:21.088940 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.088890 2557 scope.go:117] "RemoveContainer" containerID="2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094" Jul 15 20:47:21.098917 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.098899 2557 scope.go:117] "RemoveContainer" containerID="2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094" Jul 15 20:47:21.099230 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:47:21.099210 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094\": container with ID starting with 2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094 not found: ID does not exist" containerID="2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094" Jul 15 20:47:21.099310 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.099239 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094"} err="failed to get container status \"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094\": rpc error: code = NotFound desc = could not find container \"2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094\": container with ID starting with 2f4d45f6b5458b633729abe7dfae60a9cd1e61e48bca89e817fa17c0a4530094 not found: ID does not exist" Jul 15 20:47:21.112535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.112498 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:47:21.116208 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:21.116186 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-724347-54bbb76f95-ptm69"] Jul 15 20:47:22.121768 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:22.121738 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a4b2f2e-551c-42b8-8439-f9acffdc4ec0" path="/var/lib/kubelet/pods/8a4b2f2e-551c-42b8-8439-f9acffdc4ec0/volumes" Jul 15 20:47:22.122171 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:22.122113 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c83c4e4d-3119-4c18-8d12-df91d097fb22" path="/var/lib/kubelet/pods/c83c4e4d-3119-4c18-8d12-df91d097fb22/volumes" Jul 15 20:47:23.850275 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:23.850233 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:47:36.042797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.042763 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z"] Jul 15 20:47:36.043242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043177 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8e49d0-b255-4b1c-8537-0e0d73ea59c5" containerName="revoke-keys" Jul 15 20:47:36.043242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043189 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8e49d0-b255-4b1c-8537-0e0d73ea59c5" containerName="revoke-keys" Jul 15 20:47:36.043242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043209 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83c4e4d-3119-4c18-8d12-df91d097fb22" containerName="maas-api" Jul 15 20:47:36.043242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043214 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83c4e4d-3119-4c18-8d12-df91d097fb22" containerName="maas-api" Jul 15 20:47:36.043399 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043269 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8e49d0-b255-4b1c-8537-0e0d73ea59c5" containerName="revoke-keys" Jul 15 20:47:36.043399 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.043278 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83c4e4d-3119-4c18-8d12-df91d097fb22" containerName="maas-api" Jul 15 20:47:36.047741 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.047719 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:36.050473 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.050448 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qgcqm\"" Jul 15 20:47:36.056281 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.055901 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z"] Jul 15 20:47:36.189109 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.189043 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z\" (UID: \"a3d73524-420e-48ab-96b9-966758a9de68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:36.289860 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.289820 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z\" (UID: \"a3d73524-420e-48ab-96b9-966758a9de68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:36.290489 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.290467 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z\" (UID: \"a3d73524-420e-48ab-96b9-966758a9de68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:36.361187 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.361065 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:36.529395 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:36.529369 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z"] Jul 15 20:47:36.532119 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:47:36.532094 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d73524_420e_48ab_96b9_966758a9de68.slice/crio-175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9 WatchSource:0}: Error finding container 175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9: Status 404 returned error can't find the container with id 175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9 Jul 15 20:47:37.188042 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:37.188001 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" event={"ID":"a3d73524-420e-48ab-96b9-966758a9de68","Type":"ContainerStarted","Data":"17c98405b6c1ea1ad6e5f9b400cd8d301c61bbae83d2aeb6224ebe921646f716"} Jul 15 20:47:37.188535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:37.188084 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" event={"ID":"a3d73524-420e-48ab-96b9-966758a9de68","Type":"ContainerStarted","Data":"175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9"} Jul 15 20:47:37.210459 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:37.210402 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" podStartSLOduration=1.210384108 podStartE2EDuration="1.210384108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:47:37.20893388 +0000 UTC m=+2950.080927312" watchObservedRunningTime="2026-07-15 20:47:37.210384108 +0000 UTC m=+2950.082377540" Jul 15 20:47:38.196332 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:38.196295 2557 generic.go:356] "Generic (PLEG): container finished" podID="a3d73524-420e-48ab-96b9-966758a9de68" containerID="17c98405b6c1ea1ad6e5f9b400cd8d301c61bbae83d2aeb6224ebe921646f716" exitCode=0 Jul 15 20:47:38.196720 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:38.196355 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" event={"ID":"a3d73524-420e-48ab-96b9-966758a9de68","Type":"ContainerDied","Data":"17c98405b6c1ea1ad6e5f9b400cd8d301c61bbae83d2aeb6224ebe921646f716"} Jul 15 20:47:39.339801 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:39.339767 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:39.521642 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:39.521545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca\") pod \"a3d73524-420e-48ab-96b9-966758a9de68\" (UID: \"a3d73524-420e-48ab-96b9-966758a9de68\") " Jul 15 20:47:39.521950 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:39.521925 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3d73524-420e-48ab-96b9-966758a9de68" (UID: "a3d73524-420e-48ab-96b9-966758a9de68"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:47:39.622400 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:39.622361 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d73524-420e-48ab-96b9-966758a9de68-maas-api-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:40.209734 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:40.209702 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" Jul 15 20:47:40.209914 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:40.209736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z" event={"ID":"a3d73524-420e-48ab-96b9-966758a9de68","Type":"ContainerDied","Data":"175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9"} Jul 15 20:47:40.209914 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:40.209788 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="175f01a2658d5d50324e096e21baf7d30d2ce3a0ca49c47ce7cbf07dee7b71e9" Jul 15 20:47:46.421686 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.421645 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:47:46.422256 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.421884 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" podUID="f8f526d2-860f-4b51-9aed-973ac2f175dc" containerName="maas-api" containerID="cri-o://d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7" gracePeriod=30 Jul 15 20:47:46.453535 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.453499 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c"] Jul 15 20:47:46.457807 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.457774 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5292f5-29735805-ghh4c"] Jul 15 20:47:46.879290 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.879259 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:47:46.974207 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.974103 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") pod \"f8f526d2-860f-4b51-9aed-973ac2f175dc\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " Jul 15 20:47:46.974207 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.974184 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp9p7\" (UniqueName: \"kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7\") pod \"f8f526d2-860f-4b51-9aed-973ac2f175dc\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " Jul 15 20:47:46.974207 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.974206 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca\") pod \"f8f526d2-860f-4b51-9aed-973ac2f175dc\" (UID: \"f8f526d2-860f-4b51-9aed-973ac2f175dc\") " Jul 15 20:47:46.974625 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.974599 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8f526d2-860f-4b51-9aed-973ac2f175dc" (UID: "f8f526d2-860f-4b51-9aed-973ac2f175dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:47:46.976323 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.976296 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7" (OuterVolumeSpecName: "kube-api-access-xp9p7") pod "f8f526d2-860f-4b51-9aed-973ac2f175dc" (UID: "f8f526d2-860f-4b51-9aed-973ac2f175dc"). InnerVolumeSpecName "kube-api-access-xp9p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:47:46.976386 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:46.976338 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8f526d2-860f-4b51-9aed-973ac2f175dc" (UID: "f8f526d2-860f-4b51-9aed-973ac2f175dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:47:47.075553 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.075512 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f526d2-860f-4b51-9aed-973ac2f175dc-maas-api-tls\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:47.075553 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.075549 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp9p7\" (UniqueName: \"kubernetes.io/projected/f8f526d2-860f-4b51-9aed-973ac2f175dc-kube-api-access-xp9p7\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:47.075768 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.075564 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f526d2-860f-4b51-9aed-973ac2f175dc-openshift-service-ca\") on node \"ip-10-0-135-159.ec2.internal\" DevicePath \"\"" Jul 15 20:47:47.253015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.252923 2557 generic.go:356] "Generic (PLEG): container finished" podID="f8f526d2-860f-4b51-9aed-973ac2f175dc" containerID="d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7" exitCode=0 Jul 15 20:47:47.253015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.252972 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" Jul 15 20:47:47.253015 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.252995 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" event={"ID":"f8f526d2-860f-4b51-9aed-973ac2f175dc","Type":"ContainerDied","Data":"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7"} Jul 15 20:47:47.253293 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.253039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98" event={"ID":"f8f526d2-860f-4b51-9aed-973ac2f175dc","Type":"ContainerDied","Data":"54052c496f68d12f91e5b961a67f14c27b14b295517716e330558301205b80b0"} Jul 15 20:47:47.253293 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.253060 2557 scope.go:117] "RemoveContainer" containerID="d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7" Jul 15 20:47:47.263024 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.263003 2557 scope.go:117] "RemoveContainer" containerID="d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7" Jul 15 20:47:47.263364 ip-10-0-135-159 kubenswrapper[2557]: E0715 20:47:47.263339 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7\": container with ID starting with d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7 not found: ID does not exist" containerID="d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7" Jul 15 20:47:47.263467 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.263369 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7"} err="failed to get container status \"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7\": rpc error: code = NotFound desc = could not find container \"d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7\": container with ID starting with d7ce82c336b20584b4ed0703e28bd3171460d53171c055f1a3a443ea2ea949d7 not found: ID does not exist" Jul 15 20:47:47.276112 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.276060 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:47:47.282935 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:47.282902 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5292f5-74746976d8-c2r98"] Jul 15 20:47:48.122734 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:48.122699 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70cb3f86-18b0-4547-9b95-c65ca312607b" path="/var/lib/kubelet/pods/70cb3f86-18b0-4547-9b95-c65ca312607b/volumes" Jul 15 20:47:48.123124 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:48.123043 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8f526d2-860f-4b51-9aed-973ac2f175dc" path="/var/lib/kubelet/pods/f8f526d2-860f-4b51-9aed-973ac2f175dc/volumes" Jul 15 20:47:49.555035 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:47:49.554995 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zm2zh"] Jul 15 20:48:08.600465 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:08.600434 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-tbq8z_f57b7da7-fc22-43fb-b405-59083a751d46/copy-manifests/0.log" Jul 15 20:48:08.606649 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:08.606621 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-tbq8z_f57b7da7-fc22-43fb-b405-59083a751d46/manager/0.log" Jul 15 20:48:09.072514 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.072486 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735775-qx4s7_dd704b47-363c-4850-823d-626ad2643c47/cleanup/0.log" Jul 15 20:48:09.180655 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.180625 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735790-mrtzt_f79a42cd-2937-486f-b964-d43ef587005b/cleanup/0.log" Jul 15 20:48:09.291114 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.291087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735805-kqxrn_ecbf0086-15ba-4e34-b635-f15d09c4284f/cleanup/0.log" Jul 15 20:48:09.403469 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.403395 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-609ae16c-7dgcl_e582465d-544b-4157-966e-b612f16fbb5e/revoke-keys/0.log" Jul 15 20:48:09.510730 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.510700 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b3ba75ea-sxf98_09f10fc1-b2fe-4da1-9299-ecb6acbb60c7/revoke-keys/0.log" Jul 15 20:48:09.619934 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.619905 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-4fa9c7-hlqk2_01a0de51-c8bc-4368-aaca-cf1b527f829b/revoke-keys/0.log" Jul 15 20:48:09.726515 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.726433 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-44c9f6-nfxds_d5c878a4-2a47-490e-aa82-a2c2af77935c/revoke-keys/0.log" Jul 15 20:48:09.833263 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.833230 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-6dc6ee57-snc5b_79c80666-5d3e-4f1e-ac28-c749cf7de6e6/revoke-keys/0.log" Jul 15 20:48:09.940635 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:09.940601 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-b06d27-87vvb_b9178014-79e7-4892-92da-d4ae7557591a/revoke-keys/0.log" Jul 15 20:48:10.049449 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.049422 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-e45884-jl5nr_d43c3b92-15be-41b1-9ea8-8b359d195ba2/revoke-keys/0.log" Jul 15 20:48:10.156252 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.156220 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-be7332c6-slzzv_22a145f9-3553-4d9c-aea1-4f7709988566/revoke-keys/0.log" Jul 15 20:48:10.264111 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.264084 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-2bf8d569-mz5zr_a5934b82-8968-4a02-a78e-0f4d0e84ca0c/revoke-keys/0.log" Jul 15 20:48:10.368157 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.368064 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-cea3e2bc-gr96g_4b2d917d-d5ce-4790-9946-ecb8b1373fa4/revoke-keys/0.log" Jul 15 20:48:10.473238 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.473202 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-724347-vsh58_8c8e49d0-b255-4b1c-8537-0e0d73ea59c5/revoke-keys/0.log" Jul 15 20:48:10.578924 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:10.578896 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-5292f5-vmj4z_a3d73524-420e-48ab-96b9-966758a9de68/revoke-keys/0.log" Jul 15 20:48:11.448190 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.448158 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/util/0.log" Jul 15 20:48:11.454099 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.454045 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/pull/0.log" Jul 15 20:48:11.459841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.459813 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/extract/0.log" Jul 15 20:48:11.567430 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.567404 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/util/0.log" Jul 15 20:48:11.573004 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.572983 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/pull/0.log" Jul 15 20:48:11.580837 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.580818 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/extract/0.log" Jul 15 20:48:11.691731 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.691702 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/util/0.log" Jul 15 20:48:11.697571 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.697545 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/pull/0.log" Jul 15 20:48:11.703226 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.703168 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/extract/0.log" Jul 15 20:48:11.813800 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.813773 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/util/0.log" Jul 15 20:48:11.819919 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.819896 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/pull/0.log" Jul 15 20:48:11.826117 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:11.826094 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/extract/0.log" Jul 15 20:48:12.636055 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:12.636029 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zm2zh_7632d12b-fa25-4336-9b90-0081e7319920/limitador/0.log" Jul 15 20:48:13.226354 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:13.226324 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vs94g_d9a8457d-85b9-4ac8-948b-916e807a8fd2/discovery/0.log" Jul 15 20:48:13.943833 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:13.943803 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d485b46b6-r668j_7be1a52a-e535-4911-8a11-94a4d0810d99/router/0.log" Jul 15 20:48:14.506708 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:14.506673 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xqxmd_e968bcf0-aa3a-4a31-8a1c-87ab36ff4362/main/0.log" Jul 15 20:48:20.379664 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.379628 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dhp28/must-gather-6rm6n"] Jul 15 20:48:20.380036 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.379982 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d73524-420e-48ab-96b9-966758a9de68" containerName="revoke-keys" Jul 15 20:48:20.380036 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.379992 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d73524-420e-48ab-96b9-966758a9de68" containerName="revoke-keys" Jul 15 20:48:20.380036 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.380019 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f526d2-860f-4b51-9aed-973ac2f175dc" containerName="maas-api" Jul 15 20:48:20.380036 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.380025 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f526d2-860f-4b51-9aed-973ac2f175dc" containerName="maas-api" Jul 15 20:48:20.380224 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.380110 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d73524-420e-48ab-96b9-966758a9de68" containerName="revoke-keys" Jul 15 20:48:20.380224 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.380121 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f526d2-860f-4b51-9aed-973ac2f175dc" containerName="maas-api" Jul 15 20:48:20.383455 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.383432 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.386231 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.386204 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhp28\"/\"kube-root-ca.crt\"" Jul 15 20:48:20.386376 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.386323 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dhp28\"/\"default-dockercfg-kjwc9\"" Jul 15 20:48:20.387599 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.387577 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhp28\"/\"openshift-service-ca.crt\"" Jul 15 20:48:20.394937 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.394905 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/must-gather-6rm6n"] Jul 15 20:48:20.481381 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.481338 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-must-gather-output\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.481558 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.481412 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjgch\" (UniqueName: \"kubernetes.io/projected/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-kube-api-access-qjgch\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.582797 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.582757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjgch\" (UniqueName: \"kubernetes.io/projected/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-kube-api-access-qjgch\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.582982 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.582848 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-must-gather-output\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.583176 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.583160 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-must-gather-output\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.591958 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.591932 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjgch\" (UniqueName: \"kubernetes.io/projected/9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d-kube-api-access-qjgch\") pod \"must-gather-6rm6n\" (UID: \"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d\") " pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.693841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.693741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/must-gather-6rm6n" Jul 15 20:48:20.832436 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.832403 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/must-gather-6rm6n"] Jul 15 20:48:20.837595 ip-10-0-135-159 kubenswrapper[2557]: W0715 20:48:20.836360 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e869d7a_cb9f_4d7d_aa42_020c5c0d2a6d.slice/crio-6abd499eac3c59f322ac869b0958412e68644b5c62e72b06fd21887733629c4c WatchSource:0}: Error finding container 6abd499eac3c59f322ac869b0958412e68644b5c62e72b06fd21887733629c4c: Status 404 returned error can't find the container with id 6abd499eac3c59f322ac869b0958412e68644b5c62e72b06fd21887733629c4c Jul 15 20:48:20.838523 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:20.838508 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:48:21.458039 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:21.458003 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/must-gather-6rm6n" event={"ID":"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d","Type":"ContainerStarted","Data":"6abd499eac3c59f322ac869b0958412e68644b5c62e72b06fd21887733629c4c"} Jul 15 20:48:22.469656 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:22.469607 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/must-gather-6rm6n" event={"ID":"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d","Type":"ContainerStarted","Data":"785fe4f0dd532d0cdf019325d7d13f60dc9d4a02c3568ed2d8bc2410f326eb55"} Jul 15 20:48:22.470161 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:22.469670 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/must-gather-6rm6n" event={"ID":"9e869d7a-cb9f-4d7d-aa42-020c5c0d2a6d","Type":"ContainerStarted","Data":"aa72629bc17fb026a22044d9bd8d6e25d1d0d5e6463672cf4762f6e0639bb51e"} Jul 15 20:48:22.506707 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:22.506636 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dhp28/must-gather-6rm6n" podStartSLOduration=1.602641738 podStartE2EDuration="2.506616251s" totalImagesPullingTime="903.974513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:48:22.504236215 +0000 UTC m=+2995.376229648" watchObservedRunningTime="2026-07-15 20:48:22.506616251 +0000 UTC m=+2995.378609688" Jul 15 20:48:23.374973 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:23.374937 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mdxkm_186e9fb1-120a-4998-8d07-1cf46a96d6eb/global-pull-secret-syncer/0.log" Jul 15 20:48:23.546684 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:23.546656 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mfk9l_fece5b48-ad3e-4bc2-90f1-d489c87e29aa/konnectivity-agent/0.log" Jul 15 20:48:23.586792 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:23.586751 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-159.ec2.internal_19f77dbb92f973ab7c41f31009ab9ff2/haproxy/0.log" Jul 15 20:48:28.176197 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.176141 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/extract/0.log" Jul 15 20:48:28.214873 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.214845 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/util/0.log" Jul 15 20:48:28.235394 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.235366 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kjrq2_5233b00a-b63d-4f1e-a83f-77d0d3d77e12/pull/0.log" Jul 15 20:48:28.265402 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.265343 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/extract/0.log" Jul 15 20:48:28.295922 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.295874 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/util/0.log" Jul 15 20:48:28.344774 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.344732 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05qzvg_fcd89395-ec78-4cc1-9107-2220645641b6/pull/0.log" Jul 15 20:48:28.386457 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.386413 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/extract/0.log" Jul 15 20:48:28.402730 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.402496 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/util/0.log" Jul 15 20:48:28.430181 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.430095 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:48:28.436988 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.436955 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xklmz_0c136809-eac3-488a-b6a7-30b3153ac1ee/pull/0.log" Jul 15 20:48:28.452298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.452272 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:48:28.457625 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.457604 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:48:28.473691 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.473659 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77qz4_1b6cbcd2-823e-470b-aab6-1d3e6f95723a/ovn-acl-logging/0.log" Jul 15 20:48:28.489276 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.489241 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/extract/0.log" Jul 15 20:48:28.517308 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.517270 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/util/0.log" Jul 15 20:48:28.537056 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:28.537028 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef198n88_3ded2fff-f523-4165-beeb-2d73b4355624/pull/0.log" Jul 15 20:48:29.465892 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:29.465848 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zm2zh_7632d12b-fa25-4336-9b90-0081e7319920/limitador/0.log" Jul 15 20:48:31.202298 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.202273 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/alertmanager/0.log" Jul 15 20:48:31.252038 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.251994 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/config-reloader/0.log" Jul 15 20:48:31.312248 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.312209 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/kube-rbac-proxy-web/0.log" Jul 15 20:48:31.364805 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.364776 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/kube-rbac-proxy/0.log" Jul 15 20:48:31.404397 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.404371 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/kube-rbac-proxy-metric/0.log" Jul 15 20:48:31.427592 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.427558 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/prom-label-proxy/0.log" Jul 15 20:48:31.451181 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.451131 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5e26c5aa-58e2-4298-8f4b-42bb118d2dd7/init-config-reloader/0.log" Jul 15 20:48:31.502564 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.502461 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-v7kjd_5415a430-c08c-4476-b46c-b0a059e27d0b/cluster-monitoring-operator/0.log" Jul 15 20:48:31.628945 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.628887 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-s4j9d_64042cf5-0a53-4c15-8c85-e9522bbc4f25/monitoring-plugin/0.log" Jul 15 20:48:31.807092 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.807041 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ps4m9_dc291c77-b425-499b-bb76-b5226720249e/node-exporter/0.log" Jul 15 20:48:31.828544 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.828513 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ps4m9_dc291c77-b425-499b-bb76-b5226720249e/kube-rbac-proxy/0.log" Jul 15 20:48:31.848961 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.848928 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ps4m9_dc291c77-b425-499b-bb76-b5226720249e/init-textfile/0.log" Jul 15 20:48:31.875347 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.875315 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wldzk_1a10514e-4a04-4c14-affb-39370ff32e41/kube-rbac-proxy-main/0.log" Jul 15 20:48:31.893841 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.893810 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wldzk_1a10514e-4a04-4c14-affb-39370ff32e41/kube-rbac-proxy-self/0.log" Jul 15 20:48:31.918896 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:31.918862 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wldzk_1a10514e-4a04-4c14-affb-39370ff32e41/openshift-state-metrics/0.log" Jul 15 20:48:32.186872 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:32.186752 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597ffb665-qkwl6_b6c72266-38cd-4dc7-9ec8-dc28672aef2b/telemeter-client/0.log" Jul 15 20:48:32.205935 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:32.205904 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597ffb665-qkwl6_b6c72266-38cd-4dc7-9ec8-dc28672aef2b/reload/0.log" Jul 15 20:48:32.227775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:32.227747 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597ffb665-qkwl6_b6c72266-38cd-4dc7-9ec8-dc28672aef2b/kube-rbac-proxy/0.log" Jul 15 20:48:33.512317 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.512275 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq"] Jul 15 20:48:33.518870 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.518837 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.530912 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.530878 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq"] Jul 15 20:48:33.535709 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.535665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-lib-modules\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.535897 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.535716 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-sys\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.535897 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.535810 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xckkl\" (UniqueName: \"kubernetes.io/projected/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-kube-api-access-xckkl\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.535897 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.535838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-proc\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.535897 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.535886 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-podres\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.636590 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.636543 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-lib-modules\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.636861 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.636837 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-sys\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.637092 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.637053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xckkl\" (UniqueName: \"kubernetes.io/projected/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-kube-api-access-xckkl\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.637242 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.637225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-proc\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.637377 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.637362 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-podres\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.637657 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.637639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-podres\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.637877 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.637860 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-lib-modules\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.638028 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.638013 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-sys\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.638556 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.638532 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-proc\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.649835 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.649806 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xckkl\" (UniqueName: \"kubernetes.io/projected/63f68320-caf9-4cc3-8ce6-1c8aaff9f193-kube-api-access-xckkl\") pod \"perf-node-gather-daemonset-8zdnq\" (UID: \"63f68320-caf9-4cc3-8ce6-1c8aaff9f193\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:33.832275 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:33.832228 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:34.037137 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.036793 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq"] Jul 15 20:48:34.192337 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.192306 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/1.log" Jul 15 20:48:34.198327 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.198298 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-b72vx_7f9df7fe-f6bd-4e80-8d9d-578a1bba8de4/console-operator/2.log" Jul 15 20:48:34.578607 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.578569 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" event={"ID":"63f68320-caf9-4cc3-8ce6-1c8aaff9f193","Type":"ContainerStarted","Data":"3f233e14dd8e4a9af27b057b994e69b348ee69124912b89ad3715b14041157c1"} Jul 15 20:48:34.578994 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.578639 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:34.578994 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.578661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" event={"ID":"63f68320-caf9-4cc3-8ce6-1c8aaff9f193","Type":"ContainerStarted","Data":"790164048f5b5dfa6454d4bd30e31a8202637b6d3600737dc8e74b0b62ebe809"} Jul 15 20:48:34.596348 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.596280 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" podStartSLOduration=1.596258689 podStartE2EDuration="1.596258689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:48:34.595844784 +0000 UTC m=+3007.467838208" watchObservedRunningTime="2026-07-15 20:48:34.596258689 +0000 UTC m=+3007.468252123" Jul 15 20:48:34.643923 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:34.643893 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d8d6f94db-fts7f_d95f1711-64d1-42ec-9795-e479c7e97902/console/0.log" Jul 15 20:48:35.259336 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:35.259297 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-qcdzc_e4480f9f-c1e8-4483-848a-203f39ec0a92/volume-data-source-validator/0.log" Jul 15 20:48:36.344707 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:36.344679 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w5zsr_fbcfe517-551b-4abd-9dd0-b1fab5ddb979/dns/0.log" Jul 15 20:48:36.365305 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:36.365273 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w5zsr_fbcfe517-551b-4abd-9dd0-b1fab5ddb979/kube-rbac-proxy/0.log" Jul 15 20:48:36.417237 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:36.417197 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2pzf_d5e70159-25d3-43fd-a5da-aa768351e411/dns-node-resolver/0.log" Jul 15 20:48:37.025897 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:37.025868 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ql84l_a265d439-db82-4192-a282-098cffafd784/node-ca/0.log" Jul 15 20:48:38.138722 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:38.138684 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vs94g_d9a8457d-85b9-4ac8-948b-916e807a8fd2/discovery/0.log" Jul 15 20:48:38.390258 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:38.390180 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d485b46b6-r668j_7be1a52a-e535-4911-8a11-94a4d0810d99/router/0.log" Jul 15 20:48:39.081650 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.081613 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5tcf7_8a688de2-5672-448e-962b-6ea2707ab003/serve-healthcheck-canary/0.log" Jul 15 20:48:39.611920 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.611889 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rhkwb_046bd0a4-42a9-4c26-a779-dd08c9aa8d4b/insights-operator/1.log" Jul 15 20:48:39.612986 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.612962 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rhkwb_046bd0a4-42a9-4c26-a779-dd08c9aa8d4b/insights-operator/0.log" Jul 15 20:48:39.798668 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.798636 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sfd9_b78b0486-62b8-4cf0-97d8-fd8682d6604a/kube-rbac-proxy/0.log" Jul 15 20:48:39.822606 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.822573 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sfd9_b78b0486-62b8-4cf0-97d8-fd8682d6604a/exporter/0.log" Jul 15 20:48:39.849028 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:39.848987 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sfd9_b78b0486-62b8-4cf0-97d8-fd8682d6604a/extractor/0.log" Jul 15 20:48:40.599995 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:40.599967 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-8zdnq" Jul 15 20:48:42.865447 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:42.865418 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-tbq8z_f57b7da7-fc22-43fb-b405-59083a751d46/manager/0.log" Jul 15 20:48:42.890285 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:42.890262 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-tbq8z_f57b7da7-fc22-43fb-b405-59083a751d46/copy-manifests/0.log" Jul 15 20:48:44.200775 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:44.200728 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vs94g_d9a8457d-85b9-4ac8-948b-916e807a8fd2/discovery/0.log" Jul 15 20:48:44.282303 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:44.282270 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67d9958876-snvtg_4e75d5ff-0fba-4ef5-99d8-55bfe5299c9d/manager/0.log" Jul 15 20:48:44.375366 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:44.375334 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d485b46b6-r668j_7be1a52a-e535-4911-8a11-94a4d0810d99/router/0.log" Jul 15 20:48:45.785647 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:45.785622 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vs94g_d9a8457d-85b9-4ac8-948b-916e807a8fd2/discovery/0.log" Jul 15 20:48:45.972993 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:45.972956 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d485b46b6-r668j_7be1a52a-e535-4911-8a11-94a4d0810d99/router/0.log" Jul 15 20:48:49.980730 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:49.980695 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-hpnzf_e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8/kube-storage-version-migrator-operator/1.log" Jul 15 20:48:49.981519 ip-10-0-135-159 kubenswrapper[2557]: I0715 20:48:49.981499 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-hpnzf_e7514dbf-3fe1-4b48-9c9b-0cda2b6a5ad8/kube-storage-version-migrator-operator/0.log"